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PARTI 


EXECUTIVE SUMMARY 




NASA/FAA HELICOPTER SIMULATOR WORKSHOP 


PART I: EXECUTIVE SUMMARY 


William E. Larsen,* Robert J. Randle, Jr.,** Richard S. Bray,** and John Zuk+ 

Ames Research Center 


SUMMARY 


A workshop was convened by the FAA and NASA for the purpose of providing a forum at which 
leading designers, manufacturers, and users of helicopter simulators could initiate and participate in 
a development process that would facilitate the formulation of qualification standards by the regula- 
tory agency. Formal papers were presented, special topics were discussed in breakout sessions, and a 
draft FAA advisory circular defining specifications for helicopter simulators was presented and 
discussed. A working group of volunteers was formed to work with the National Simulator Program 
Office to develop a final version of the circular. The workshop attracted 90 individuals from a con- 
stituency of simulator manufacturers, training organizations, the military, civil regulators, research 
scientists, and five foreign countries. A great amount of information was generated and recorded 
verbatim. This information is presented herein within the limits of accuracy inherent in recording, 
transcribing, and editing spoken technical material. 


INTRODUCTION 


A NASA/FAA-sponsored helicopter simulator workshop was convened (23-26 April, 1991) at 
the Biltmore Hotel in Santa Clara, California. The purpose of the workshop was to support the Fed- 
eral Aviation Administration in clarifying qualification requirements for rotary-wing flight-training 
simulators and to review the draft Advisory Circular, “Helicopter Simulator Qualification,” AC 
120-XX written to implement these requirements. Funding for this and other project activities were 
provided by the the FAA’s Vertical Flight Special Programs Office, ARD-30, in support of the 
National Simulation Program Office, ASO-205. These activities are authorized and funded by 
Interagency Agreement DTFA01-88-Z-02015, Rotorcraft Simulator Technology, between the FAA 
and NASA, June 15, 1988. 

Three important purposes were identified that could best be served at a workshop consisting of 
knowledgeable and interested representatives of the training simulator community. First, the work- 
shop would provide a forum. In rotary-wing flight simulator training and technology there are many 
indeterminacies, and there is no systematic method for the formulating and resolving of questions 


*Federal Aviation Administration. 

**Sverdrup Technology, Inc., Ames Research Center. 
I'Civil Technology Office, NASA Ames Research Center. 



relating to the ability of simulators to accomplish the training required by FAA regulations (Title 14 
CFR). A primary goal of the workshop would be to elicit expert opinion and experience in an effort 
to define those questions and to cull from the attendees’ presentations and comments some guide- 
lines for an approach to their resolution. Subsequent documentation and dissemination of this infor- 
mation would make a substantive contribution to simulator qualification efforts and to the guidance 
of future research and development. 

Second, the workshop would provide a context for public preview and comment on the draft 
Advisory Circular, 120-XX, October 22, 1990, “Helicopter Simulator Qualification.” The attendees 
would be those professionals for whom the Advisory Circular is of immediate concern. They would 
be a pool of candidates for member ship on a working group, requested by the manager of the 
National Simulator Evaluation Program, that would be responsible for developing the final form of 
the Advisory Circular. 

Third, the workshop would serve to collect valuable information which would be documented 
and disseminated. The NASA/FAA simulator qualification project is not currently a research project 
with long-term goals but a circumscribed effort dependent on existing technical information, driven 
largely by the need to produce a valid and consensus Advisory Circular and training regulations. The 
workshop was conceived as a source of “data" which would (1) help in the finalization of the draft 
Advisory Circular, (2) increase the helicopter community’s awareness of and concern with training 
simulator issues, and (3) perhaps identify further research and development objectives. The results of 
the workshop would be documented in two parts, this Executive Summary, and Part II, Workshop 
Proceedings and Session Compendium. Both parts would be distributed to all attendees. 


WORKSHOP STRUCTURE 


Seventeen speakers were invited from the helicopter training simulator industry and from the 
military to present formal papers over the first day and a half of the conference. Three panel discus- 
sions (breakout sessions) were scheduled on the afternoon of the second day and three more on the 
morning of the third day. The panel session topics were: 

A. Training Limits, Allowances, Future 

B. Scene Content and Simulator Training Effectiveness 

C. Low-cost Training Alternatives: Part-and Full-Task Trainers 

D. Dynamic Response and Engineering Fidelity in Simulation 

E. Current Training: Where Are We? 

F. Aero Modelling. 

Panel sessions D and F were combined into a single session at the request of the panel members 
because of the similarity of their content and the overlapping expertise of the discussants. 

The panel moderators and participants were instructed that the intention was to promote a free- 
flowing discussion in which all contributions were welcomed and desired. At the completion of the 
breakout sessions each session moderator summarize^ the discussions that had taken place. 
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On the afternoon of the third day the manager of the National Simulator Evaluation Program, 

Mr. Ed Booth, chaired a meeting of the conference-at-large at which he invited comment on the draft 
Advisory Circular, 120-XX. He also formed a volunteer working group from the assembled partici- 
pants to meet with him at a future date to work on the finalization of the draft Advisory Circular. 

The welcoming address was presented by C. Thomas Snyder, Director of Aerospace Systems, 
NASA Ames Research Center. Mr. Snyder presented a short history of simulator development at 
Ames. 

The keynote speaker was James D. Erickson, Manager, Southwest Region Rotorcraft Directorate, 
Aircraft Certification Service, FAA. Mr. Erickson noted that rotorcraft simulation has not kept pace 
with fixed-wing simulation. He said that the military successes with simulation would be given 
attention and that the importance of developing useful, acceptable, and objective standards in the 
civil sector would be emphasized. 


PRESENTATION ABSTRACTS 


The duties of moderator of the formal-paper sessions were ably executed by Mr. James 
McDaniel, Manager, Vertical Flight Program Office, ARD-30, FAA Headquarters. Seventeen 
speakers made presentations at the workshop. Abstracts of the presentations follow. Cliff 
McKeithan’s paper, which was not originally scheduled, is also abstracted. 

1. HELICOPTER SIMULATOR STANDARDS. Edward Boothe, Manager, FAA National 
Simulator Evaluation Program. 

The initial advisory circular was produced in 1984 (AC 120-XX). It was not finalized, however, 
because the FAR’s for pilot certification did not recognize helicopter simulators and, therefore, 
permitted no credit for their use. That is being rectified, and, when the new rules are published, stan- 
dards must be available for qualifying simulators. Because of the lack of a data base to support spec- 
ification of these standards, the FAA must rely on the knowledge of experts in the simulator/training 
industry. A major aim of this workshop is to form a working group of these experts to produce a set 
of standards for helicopter training simulators. 

2. HELICOPTER SIMULATION: AN AIRCREW TRAINING AND QUALIFICATION 
PERSPECTIVE. Richard A. Bimbach, Manager, Air Carrier Training Branch, FAA Flight Standards 
Service. 

This paper reviews some of the unique considerations that distinguish the commercial rotary- 
wing domain from its fixed-wing counterpart and that should give the FAA cause to proceed cau- 
tiously in drawing upon its fixed-wing experience. A major point: device qualification should be 
accomplished in a context of an overall training and qualification system. This approach would take 
as its starting point a detailed analysis of rotary-wing missions and tasks from which proficiency 
objectives can be systematically developed. 
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3. ROTORCRAFT MASTER PLAN. Peter V. Hwoschinsky, FAA Vertical Flight Program 
Office. 

The Rotorcraft Master Plan contains a comprehensive summary of active and planned FAA ver- 
tical flight research and development. Since the Master Plan is not sufficient for tracking project 
status and monitoring progress, the Vertical Flight Program Plan will provide that capability. It will 
be consistent with the Master Plan and, in conjunction with it, will serve to ensure a hospitable envi- 
ronment if the industry presents a practical vertical-flight initiative. 

4. SIMULATORS FOR CORPORATE PILOT TRAINING AND EVALUATION. Curt 
Treichel, Manager, Training for Corporate Aircraft, United Technologies, Inc. 

Corporate aviation relies heavily on simulation to meet training and evaluation requirements. It 
appreciates the savings in fuel, money, noise, and time, and the added safety it provides. Also, simu- 
lation provides opportunities to experience many emergencies that cannot be safely practiced in the 
aircraft. There is a need to focus on the advantages of simulator training over aircraft training and to 
provide appropriate changes in the regulations to allow the community to make it possible for users 
to take full advantage of simulation. 

5. TRAINING EFFECTIVENESS ASSESSMENT: WHERE ARE WE? Greg McGowan, Man- 
ager, West Palm Beach Learning Center, FlightSafety International. 

Over 9,000 pilot training courses have been conducted at FSI using the Bell 222 and Sikorsky 
S-76 simulators. Through the use of FAA exemptions, these simulators can be used for certain train- 
ing and checking credit. The history of the development and use of commercial helicopter simulators 
and the opportunities for their increased utilization and use were explored. 

6. CURRENT TRAINING: WHERE ARE WE? Gerald Golden, Director of Training, Petroleum 
Helicopters, Inc. 

Petroleum Helicopters, Inc. maintains a staff 750 helicopter pilots. The initial, transition, 
upgrade, and recurrent training for these pilots requires a significant financial outlay. Since a major 
portion of that training is done to satisfy the requirements of FAR 61.57, “Recent Flight Experience, 
Pilot in Command" and 135.297, “Pilot in Command: Instrument Proficiency Check Requirements,” 
much could be accomplished using an approved simulator. However, it is imperative that credit be 
given for training time spent in the simulators and that the device be realistic, practical, and 
affordable. 

7. HELICOPTER SIMULATION QUALIFICATION. Brian Hampson, Director of Engineering 
Administration, CAE Electronics. 

CAE has extensive experience in building helicopter simulators and has participated in group 
working sessions for fixed-wing advisory circulars. Against this background issues that should be 
addressed in establishing helicopter approval criteria were highlighted. Some of these issues are not 
immediately obvious and may, indeed, be more important than the cri'e^a themselves. 
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8. HELICOPTER SIMULATION: MAKING IT WORK. Barry Payne, Aviation Network (NZ) 
Limited. 

The opportunities for improved training and checking by using helicopter simulators are greater 
than they are for airplane pilot training. Simulators permit the safe creation of training environments 
that are conducive to the development of pilot decision-making, situational awareness, and cockpit 
management. This paper defines specific attributes required in a simulator to meet a typical heli- 
copter operator’s training and checking objectives. 

9. HELICOPTER TRAINING SIMULATORS: KEY MARKET FACTORS. John McIntosh, 
Vice President, Hughes Simulation Systems. 

Simulators will gain an increasingly important role in training helicopter pilots only if the simu- 
lators are of sufficient fidelity to provide positive transfer of skills to the aircraft. This must be done 
within an economic model of return on investment. Although rotor pilot demand is still only a small 
percentage of overall pilot requirements, it will grow in significance. This presentation described the 
salient factors influencing the use of helicopter training simulators. 

10. TRAINING EFFECTIVENESS ASSESSMENT: METHODOLOGICAL PROBLEMS 
AND ISSUES. Kenneth Cross, President, Anacapa Sciences. 

The U.S. military uses a large number of simulators to train and sustain the flying skills of heli- 
copter pilots. Despite the enormous resources required to purchase, maintain, and use those simula- 
tors, little effort has been expended in assessing their training effectiveness. One reason for this is the 
lack of an evaluation methodology that yields comprehensive and valid data at a practical cost. Some 
of these methodological problems and issues that arise in assessing simulator training effectiveness, 
as well as problems with the classical transfer-of-leaming paradigm were discussed. 

1 1. DETERMINING THE TRANSFERABILITY OF FLIGHT SIMULATOR DATA. David 
Green, President, Starmark Corporation. 

This paper presented a method for collecting and graphically correlating subjective ratings and 
objective flight test data. The method enables flight-simulation engineers to enhance the simulator 
characterization of rotorcraft flight in order to achieve maximum transferability of simulator 
experience. 

12. PROGRESS THROUGH PRECEDENT: GOING WHERE NO HELICOPTER SIMU- 
LATOR HAS GONE BEFORE. Richard J. Adams, Vice President, Advanced Aviation Concepts. 

Helicopter simulators have been approved by means of special exemption; there are no FA A 
standards for simulators used in training or airmen certification checking. The fixed-wing industry 
provides a precedent which can be used for expediting implementation of helicopter simulators. The 
analysis in this paper is founded on the experience with that precedent and is driven by a clear defi- 
nition of helicopter user needs for (1) improved training at lower cost, (2) more comprehensive 
emergency training at lower risk, (3) increased fidelity of transition and instrument training com- 
pared with low-cost aircraft alternatives, and (4) certification credit for improved simulator training. 
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13. TRANSFER OF TRAINING AND SIMULATOR QUALIFICATION. Jack Dohme, 
Research Scientist, U.S. Army Research Institute. 

Transfer of training studies at Fort Rucker using the backward-transfer paradigm have shown 
that existing flight simulators are not entirely adequate for meeting training requirements. Using an 
ab initio training research simulator, a simulation of the UH-1, training effectiveness ratios were 
developed. The data demonstrate it to be a cost-effective primary trainer. A simulator qualification 
method was suggested in which a combination of these transfer-of-training paradigms is used to 
determine overall simulator fidelity and training effectiveness. 

14. VALIDATION AND UPGRADING OF PHYSICALLY BASED MATHEMATICAL 
MODELS. Ronald Du Val, President, Advanced Rotorcraft Technology. 

The validation of the results of physically-based mathematical models against experimental 
results was discussed. Systematic techniques are used for: (1) isolating subsets of the simulator 
mathematical model and comparing the response of each subset to its experimental response for the 
same input conditions; (2) evaluating the response error to determine whether it is the result of incor- 
rect parameter values, incorrect structure of the model subset, or unmodeled external effects of cross- 
coupling; and (3) modifying and upgrading the model and its parameter values to determine the most 
physically appropriate combination of changes. 

15. FREQUENCY RESPONSE TECHNIQUES FOR DOCUMENTATION AND IMPROVE- 
MENT OF ROTORCRAFT SIMULATORS. Mark Tischler, Rotorcraft Group Leader, Army 
Aeroflightdynamics Directorate, Ames Research Center. 

Pilot-in-the-loop characterizations are most naturally formulated in terms of end-to-end fre- 
quency responses, so a frequency-response-based method is the natural approach to assessing simu- 
lator dynamic fidelity. A comprehensive frequency-response approach used heavily by Ames 
Research Center researchers was described, and results were presented from a number of simulator 
fidelity assessment studies. Those studies included UH-60 mathematical model validation and 
upgrade, ASTOVL linear model extraction, and documentation of the Vertical Motion Simulator (at 
Ames Research Center) motion and visual system characteristics. 

16. BANDWIDTH AND SIMDUCE AS SIMULATION FIDELITY CRITERIA. David Key, 
Chief, Flight Control Branch, Army Aeroflightdynamics Directorate. Ames Research Center. 

The potential application of two concepts from the new Handling Qualities Specification for 
Military Rotorcraft was discussed. The first concept is bandwidth, a measure of the dynamic 
response to control. The second is a qualitative technique developed for assessing the visual cue 
environment the pilot has in bad weather and at night. SIMulated Day Usable Cue Environment 
(SIMDUCE) applies this concept to assessing the day cuing fidelity in the simulator. 

17. METHODOLOGY DEVELOPMENT FOR EVALUATION OF SELECTIVE FIDELITY 
ROTORCRAFT SIMULATION. Cliff McKeithan, Georgia Institute of Technology. (Authors: 

Major William D. Lewis, Dr. D.P. Schrage, Dr. J.V.R. Prasad, Major Daniel Wolfe). 
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This paper addressed the initial step toward the goal of establishing performance and handling 
qualities acceptance criteria for realtime rotorcraft simulators through a planned research effort to 
quantify the system capabilities of “selective fidelity" simulators. Within this framework the simula- 
tor is then classified based on the required task. The simulator is evaluated by separating the various 
subsystems (visual, motion, etc.) and applying corresponding fidelity constants based on the specific 
task. This methodology not only provides an assessment technique, but also provides a technique to 
determine the required levels of subsystem fidelity for a specific task. 


COMMENTARY 


The workshop presentations and discussions evoked a broad range of pertinent background and 
experiential information, problem definitions, and problem solution guidelines. Some of the more 
significant of these are summarized below. 

1. There appears to be a ready worldwide market for simulators and training devices. Although 
the military has hundreds of simulators, little has been done in the civilian market as a result of lack 
of enabling legislation for helicopter simulators. There are only two helicopter simulators in the 
United States that have been provisionally approved by the FAA, the Bell PH 222 and the Sikorsky 
S-76B. These are approved by exception for considerable credit toward pilot certification, but the 
pilot must still pass a checkride in the helicopter. These simulators are sophisticated devices in terms 
of, for example, their dynamic models and motion systems. At the other end of the market lies a 
generic, fixed-base training device that can be used to teach and review all of the visual helicopter 
flight maneuvers and techniques, along with systems functionality and navigation. This type of 
device offers the manufacturer the most flexibility in providing all of the helicopter fidelity and 
functionality at the lowest cost without having to comply with FAA AC- 120 XX. 

2. A report (Abstract #9) of a survey of the simulator market indicated that only eight so-called 
high-end ($12 million to 25 million) simulators are needed worldwide over the next decade. Many 
more (100 to 200) lower-end devices ($1.0 million to 1.2 million) could be supported. Rotary-wing 
training managers emphasized this in their desire for approval of less expensive part-task training 
devices in earning credits toward meeting regulatory requirements. In addition to helicopter training 
simulators, the industry and government should move out on issues related to tilt-rotor/wing and the 
regulations, infrastructure, and technology issues that will be of consequence in the mid to late 
1990’s. Timing of FAA action is consistent with market forecasts and the needs of helicopter 
operators. 


3. There was a general feeling that the full capability of current helicopter simulators was not 
being exploited owing, perhaps, to some hesitancy on the part of the authors of regulatory require- 
ments. The regulations and exemptions as they stand today still discourage industry from using the 
simulator to its fullest potential. Many maneuvers and emergency procedures cannot be safely done 
in the aircraft but can be done safely, repetitively, quickly, and economically in a simulator. Thus, a 
desire was expressed to expand the uses of simulators to allow credit for the training of tasks from 
more emergency procedures through instrument ratings to crew coordination and resources 
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management. As one attendee stated: “the couple of things that cannot be done well in the simulator 
are nothing compared with the many things that cannot be done [at all] in the aircraft." Also, the 
more that credit is withheld for training and checking done in simulators the more it is a disincentive 
to use them. 

4. A boost in support of helicopter simulator utilization will be provided by a new proposed rule, 
NPRM, Part 142, which will authorize and regulate Certificated Training Centers. The objectives 
of the new rule are to increase simulator use, eliminate simulator exemptions, standardize training, 
and standardize FAA oversight of trainers through a centralized, national training program approval 
process. The new rule will cross air-carrier and non-air-carrier lines, and no distinction is made 
between fixed-wing and helicopter simulators. In an effort to maintain a broad perspective the rule 
would not specify in any detail differences in use of helicopter and fixed-wing simulators. Rather, 
the FAA would issue a certificate to the training center based on a set of training specifications 
which could be changed much easier than changing the certification. Part 142 will train to existing 
standards of Parts 61, “Certification: Pilots and Flight Instructors,” 121, “Certification and Opera- 
tions: Domestic, Flag, and Supplemental Air Carriers and Commercial Operators of Large Aircraft,” 
and 135, “Air Taxi Operators and Commercial Operators,” and may be expandable to Parts 63, 
“Certification: Flight Crewmembers Other Than Pilots,” 133, “Rotorcraft External Load Opera- 
tions,” 137, “Agricultural Aircraft Operations" and possibly others. Parts 121 and 135 operators 
contracting with Part 142 training centers would not have to duplicate any part of the training 
program. Part 142 will either replace Part 141, “Pilot Schools,” or complement it, in which case 
Part 142 schools would cooperate with Part 141 schools in finding mutually satisfactory arrange- 
ments for training students. This alternative will be presented in the NPRM and comments will be 
solicited. 

5. There is a dichotomy of training philosophy and opinion with respect to the amount of credit 
that should be allotted to simulators as opposed to aircraft for skill demonstration. The regulatory 
agencies, though enthusiastic and motivated to grant approval for more simulator training and check- 
ing, must proceed cautiously, supported by empirical evidence, in the interest of safety, and training 
relevancy, and in view of potential liability in a litigious society. On the other hand, industry is also 
enthusiastic about increased use of simulators in lieu of the aircraft in the interest of safety, rele- 
vancy, and economy, and its representatives point out that these interests, based on their experience, 
will be better served by more use of simulators that offer much more versatility in terms of task and 
maneuver repertoire than can the aircraft. The aircraft, as a training device, is severely limited. 
Exemptions should still be sought in the interim before the publication of Part 142 (perhaps late 
1992). 

6. At a seminar in 1987, Vertical Flight Training Needs and Solutions, co-sponsored by the FAA 
and the Helicopter Association International, it was determined that human error-related accidents 
were the greatest problem the helicopter industry faced. Up to 80% of all helicopter accidents were 
in one way or another caused by human error, not by deficiencies in flying or control skills, and 
simulator training along with decision-making training was seen as an effective way to help reduce 
this kind of accident. The helicopter pilot frequently is under the pressure of a high workload situa- 
tion and engages in a variety of industrial-commercial tasks, such as sling operations, flying crane, 
airtaxi, offshore oil platform work, high-altitude slope work, and cattL/wildlife herding that are 
inherently difficult and potentially dangerous. These are seldom “canned” or routine maneuvers and 
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therefore require good decision-making and judgmental skills. This kind of training, reminiscent of 
line-oriented flight training and cockpit resources management training in the fixed-wing world, 
requires neither high dynamic fidelity nor a type-specific simulator; a generic model (low-end cost) 
would be more than sufficient. The expanded use of and increased FAA credit for training in more 
generic simulators (training devices) was a pervasive issue at the workshop (see below). 

7. The expanded use for credit of simulators in fixed-wing training, which has been successful 
under the Part 121, Appendix H, Advanced Simulation Plan, was frequently referred to. However, 
the application of fixed-wing simulator technology to rotary-wing training has some drawbacks. The 
different maneuvering capabilities of helicopters with omni-directional flight in proximity to the 
ground appears to require more capability in the visual scene than is currently available. The com- 
plaints are that the simulated visual cues do not adequately support veridical perception of altitude 
and altitude rates. This is mainly attributed to lack of good textural cues and to restricted fields of 
view, particularly in the downward direction, since the ground plane must be extrapolated by the 
trainee from the forward-oriented, perspective-drawn visual scene. Other attendees felt that existing 
simulators do give effective training down to the ground. These opinions probably should be tem- 
pered by consideration of the kind and skill level of the training being given. This caveat would 
appear to apply to all discussions of the contribution to training of all the simulator subsystems. 

8. Physical simulator fidelity is desirable, but functional fidelity (training effectiveness) should 
be the goal. The lack of a systematic method within the civil rotary-wing community for determining 
simulator cost and training effectiveness makes it difficult to predict the levels of fidelity that are 
required for meeting (or exceeding) training performance and regulatory standards except through 
user experience; this is a long-term, unsystematic, and possibly biased process. 

9. Current simulator design is hardware technology-driven. However, high fidelity of individual 
components of the simulator such as handling qualities, motion, and the visual scene does not of 
itself guarantee high training effectiveness. In the absence of discriminatory data, the effort to pro- 
vide high fidelity is a current default position based on inferential logic. The proof of simulator effi- 
cacy is transfer-of-training from simulator to aircraft at reasonable savings and return on investment. 
Controlled studies of these outcomes in the civil community are neither available nor planned. 

10. Training industry representatives expressed interest in joining NASA and the FAA in 
addressing the issue of transfer-of-training studies as a screening strategy for the selection of behav- 
iors trainable in simulators for credit. This could be done by using current training facilities, training 
personnel, and trainee pools. None of these three potential participants (training industry, NASA, 
FAA) currently has a unique capability in the area of formal transfer-of-training and training assess- 
ment studies of large populations but probably could share in the planning, cost, management, and 
technology applications of such efforts. 

11. Our current difficulty in relating engineering simulator fidelity to training excellence also 
presents difficulties in the specification of test values and tolerances for the proposed advisory circu- 
lar for simulator qualification. 

12. It is recognized that the body of descriptive data obtained during the development of a 
helicopter is rarely adequate for definition of an accurate simulation model. Later flight tests to 
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gather the necessary data tend to be very expensive. The FAA prefers that these data be generated by 
the manufacturers of the aircraft, but simulator manufacturers have on occasion relied on third-party 
tests. The absence of data necessarily increases reliance on pilots’ subjective assessments. 


13. NASA and the military have been making increased use of a flight-testing technique known 
as “frequency-sweep” that produces data, at modest cost in flight time, that is well-conditioned for 
use in helicopter modelling. The technique can be applied to the complete simulator, including 
motion and visual systems, for comprehensive verification of simulator fidelity. 

14. Blade-element rotor modelling was recommended as the way to insure the fidelity of the 
simulator dynamic response and a strong point was made regarding the rapidly decreasing cost of 
computer capacity to accommodate such models. Again, because of uncertainties in the description 
of the aircraft, and the uncertain correlation between dynamic fidelity and training efficiency, this 
position was contested to some degree. Less complex models cannot be discarded out-of-hand until 
more evidence is available that the added complexity is training-justified. 

15. There appears to be a consensus that a maximum visual scene and cockpit motion transport 
delay of 100 msec is a realistic specification for helicopter simulators. This more rigorous constraint 
than imposed on fixed-wing trainers reflects the higher control band-width typical of helicopters. 

1 6. The value of expensive and complex motion systems is questioned when their contribution 
to training is considered. A bad motion system is worse than no motion system at all, and the 
contribution of a motion system to training may be highly task-dependent. The research literature 
seems to support this position, but, other than for the advantageous cueing of disturbance motion 
over simple maneuver motion, it has not been determined which sets of tasks can be better trained 
using motion cueing. Simulation of the vibration modes is recognized as a valuable contributor to 
simulator subjective fidelity. 


17. The need for a wide field-of-view and abundant scene detail in simulation of hover tasks is 
recognized. It is also recognized that the visual simulation represents about half the cost of a modem 
simulator. This cost is especially high if the two crew members are provided with equivalent fields. 
Particularly, considering some of the new lower-cost visual systems being demonstrated, there exists 
a strong challenge to develop more cost-effective systems identified by a careful assessment of 
training needs and aircraft/simulator training time trade-off. 

18. Collimation of visual scenes, a source of increasing simulator initial cost and upkeep, may 
be of questionable value in comparison with real image displays. They do provide a dramatic illusion 
of great distance and of a large “gaming” area; however, the localization of ah picture elements at 
optical infinity leads to perceptual difficulties in estimating size and distance at short ranges, say, 

10 ft (wheels on ground) to 50 ft (hover), the crucial range for helicopter maneuvering near the 
ground. This effect, coupled with limitations in the downward field of view and texture, make it 
difficult to localize the ground plane and to perceive altitude and altitude rates. 
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19. It is recognized that because of visual and motion cueing limitations, simulated tasks, 
particularly those in proximity to the terrain, are harder to fly than the real task, even if the aircraft 
model itself is of very high fidelity. The extent to which this presents an obstacle to effective use of 
the simulator was the subject of brisk discussion. Some voices supported the addition of compensa- 
tion (for example, stability augmentation) to effect a more realistic work load in the critical tasks. 

20. The hearing session on the draft of the Advisory Circular 120-XX for the qualification of 
helicopter simulators was cooperative rather than contentious, probably a result of the wide latitude 
given to industry participants in voicing their viewpoints throughout the previous 2 days of the 
workshop. Mr. Boothe had no problem in recruiting 30 volunteers to make up a panel to assist him in 
the further refinement and finalization of the circular. Several areas for further review were sug- 
gested and will be pursued by the volunteer working group. They seemed united on the need for the 
proposed advisory circular and the enabling FAA regulation; and the draft circular appeared to be a 
workable document for their efforts. The first meeting of the panel was scheduled for 23-25 July, 
1991 in West Palm Beach, Florida. 


EPILOGUE 


The individuals of the NASA/FAA project team who were responsible for the inception, organi- 
zation, and execution of the Helicopter Simulation Workshop are indebted to panel moderator David 
A. Lombardo for his unsolicited reflection of their intentions. In part, he said: 

In the early days of aviation the designer was the trainer and the user. Most things were done by 
trial and error, including aircraft design, pilot certification and standards, and pilot training. . . . For- 
tunately, that trend is slowly changing with the old guard passing the torch to new, better technologi- 
cally informed replacements. The new emphasis is on user involvement in the initial design of hard- 
ware, software, and liveware training and certification. This symposium is an example of that 
emerging trend.” 

The NASA/FAA project team members would like to extend to all workshop participants a very 
sincere expression of gratitude for their involvement in the workshop. Your enthusiasm and willing- 
ness to take responsibility for the future of simulator training by bringing your expertise to bear on a 
difficult technological area is greatly appreciated. We will extend a modest retum-of-favor by deliv- 
ering to you in a timely manner the planned workshop documentation. 
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MESSAGE FROM THE CONFERENCE CHAIRMAN 


BILL LARSEN 


I wish to thank you for participating in the recent 
Helicopter Simulator Workshop and for making it such a 
success. Without your contribution the workshop would 
not have been possible. 

It is clear, considering recent advances in training 
simulator technology and your statements during the 
workshop, that we will see enabling legislation that will 
provide increased credit for ground-based training. To 
some extent, this is already taking place, as reflected by 
the proposed rule making of Part 142 Title 14 CFR enti- 
tled “Certification Training Centers,” the National Simu- 
lator Program Offices’ Draft Advisory Circular 
No. 120-XX, “Helicopter Simulator Qualification,” the 
recently published special FAR 58 “Advanced 
Qualification Program,” and the FAA’s National Plan for 
Aviation Human Factors. 

For reasons, the simulator has become the aircrew 
training and checking tool of choice. This view was very 
apparent at the recent workshop. Along with the advances 
in simulator and training equipment technology has come 
an increasing awareness of the need for a systematic 
approach to device and training system design and speci- 
fication. The emerging realization is that simulators and 
training devices are more than just an example of modern 
engineering technical excellence: they are quintessentially 
devices for the enhancement of human behavior. 

The FAA certifies personnel, equipment, and pro- 
cedures. The equipment certified includes aircraft, simula- 
tors (aircrew training/checking devices), and other equip- 
ment used in the NAS. Traditionally, the FAA has quali- 
fied flight simulators on the basis of engineering criteria 
that reflect the extent to which the characteristics of a 
given system are equivalent to those of the aircraft. Train- 
ing transfer effectiveness — the extent to which an individ- 
ual who meets a standard of proficiency in the simulator 


can be expected to exhibit a known level of proficiency in 
the aircraft — has been assumed. This approach has proved 
satisfactory for high-fidelity simulations, but it is appro- 
priate that additional factors be considered in establishing 
qualification criteria for training devices that rank lower 
on the physical fidelity continuum. 

The FAA regulatory mission requires a sound basis 
for qualifying such equipment in training program and 
airman certification applications. Operators have been 
encouraged by the advanced qualification program to be 
innovative in designing training systems and equipment. 
Equipment used to establish or to maintain currency must 
be evaluated and approved against a set of criteria estab- 
lished by the FAA administrator for a particular qualifica- 
tion level. In this regard it is imperative that research be 
conducted to establish scientifically solid evaluation crite- 
ria that will be applicable to all such devices subject to 
FAA qualification. 

The program’s goal is to determine what level of 
simulator or training device is necessary to achieve a 
given training objective so that an aircrew member can 
qualify for credit toward regulated flight training. The 
amount of simulator training that is necessary to satisfy 
flight training requirements currently is determined by 
regulation. The regulation reflects the assumption that the 
more realistic the simulator the greater the value of the 
training. However, the level of the fidelity of represented 
parameters (e.g., visuals, handling qualities, motion) that 
is required to satisfy these regulations has not been empir- 
ically determined. 

The transcribed and edited versions of the speaker’s 
presentations follow. Summary statements of the separate 
panel discussions and a list of workshop attendees appear 
as appendixes. 
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Bill Larsen served as a test pilot in the Air force and worked for 27 years in the 
aerospace and computer industries and with NASA. During that period, Mr. Larsen 
participated in R&D programs related to military and commercial transport aircraft 
and various missile systems, and served as engineering director for the design and 
development of a main frame computer. At NASA, he developed and conducted 
flight experiments for the Apollo spacecraft. Since joining the FAA in 1974, 

Mr. Larsen’s work has encompassed cockpit alert and warning systems, an ATC 
simulation system, digital avionics systems, and fault-tolerant digital aircraft sys- 
tems. In addition, he has conducted extensive investigations into the effects of elec- 
tromagnetic threats to aircraft systems, including fly-by-wire and fly-by-light digital 
flight control systems. Mr. Larsen has served on several technical committees, and 
has participated in and organized several Digital Avionics Systems Conferences 
sponsored by the IEEE and AIAA. He has a B.S. degree in mechanical engineering 
and B.S. and M.S. degrees in electrical engineering from the University of 
Washington. 
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WELCOME 


C. THOMAS SNYDER 


Good morning, ladies and gentlemen. It is a pleasure 
for me to welcome you to this Helicopter Simulator 
Workshop on behalf of NASA and the FAA. I am sure 
that many of you in the audience are aware that the rela- 
tionship between NASA and the FAA here at Ames 
Research Center has been a very strong one over the 
years, especially in simulation. The purpose of the work- 
shop today is to assess the state of simulation technology, 
especially that of helicopter simulation, and to define a 
path leading to the qualification of helicopter training 
simulators. We see NASA’s role in this process as one of 
support to the FAA, and we are pleased to be a part of this 
process in that sense. We believe that it has important 
implications for the entire vertical flight community. 

Now, I mentioned this very close relationship 
between NASA and the FAA. I have had a first-hand 
involvement in that activity, especially during my early 
years as a researcher at Ames. So I hope you will bear 
with me while I reminisce a little about the changes we 
have seen in simulation over 25 short years. 

The genesis of the NASA-FAA relationship really 
goes back to the early 1960s here at Ames and to a very 
visionary and a very energetic FAA employee from the 
Western Region named Joe Tymczyszyn. I am sure many 
of you know Joe. It is with a really warm spot in my heart 
I remember Joe predicting how simulators could be 
applied to expedite and simplify the certification of new 
classes of aircraft, to understand their operating character- 
istics before they really became hardware, and to get a 
jump on the process. I remember, too, the energy he put 
into pursuing that goal, as a result of which the NASA- 
FAA research program was established. 

One of our first activities was to set up a simulation 
of a DC-8 to validate the idea; a kind of mock certification 
was conducted with that simulation. The hardware was 
basically a fixed-base transport cockpit with a rather crude 
single- window external visual display (fig. 1). 

The display was generated by a moving-belt model 
runway viewed by a servo-driven TV camera that created 


a black and white picture projected onto a screen, and 
viewed through a collimating lens (fig. 2). I remember all 
the trouble we had keeping the servo system tuned up to 
do a job that was more than it was designed for. From that 
relatively successful experiment we moved on to examine 
supersonic transport flight characteristics and the certifi- 
cation criteria related to them. 

In 1969, the large-motion-base six-degrees-of- 
freedom Flight Simulator for Advanced Aircraft, or FSAA 
as we came to call it, was commissioned (fig. 3). With 
that came digital computation (up to that time we used 
analog computation) and much improved visual displays 
of the model terrain-board type. The very large (±50 ft) 
lateral travel of this simulator was excellent for studying 
lateral directional characteristics of large aircraft and for 
studying the effects of engine failure. So there was a lot 
of work done in those areas. 

Following the demise of the U. S. Supersonic Trans- 
port program, attention was directed toward Concorde 
certification, and a very successful program was con- 
ducted with the joint authorities that contributed to the 
special conditions for the Concorde. It was also during 
that time frame that the FAA decided to establish a field 
office at Ames, and that office has continued to this day. 
The certification criteria simulation work was then 
directed to questions related to the introduction of wide 
bodies, the Boeing 747, and later to STOL certification 
criteria. The FSAA was also used during that period in the 
competitive evaluation of the proposals leading to the 
XV- 15 tilt-rotor research aircraft. That was the first such 
use of a simulator, to my knowledge 

In 1980 the six-degrees-of-freedom Vertical Motion 
Simulator (VMS) was introduced (fig. 4). It has ±30 ft of 
vertical travel, ±20 ft of lateral travel and six degrees of 
freedom. It is the real workhorse of our activity today. 

It was also at about that time that we transitioned to 
computer-generated visual displays and multi-window 
external scenes. In addition to continuing research on 
powered-lift STOL and VSTOL aircraft using this 
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Figure 1. Fixed-base transport simulator. 
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Figure 2. Moving-belt visual image generator. 







Welcome 



Figure 3. Flight simulator for advanced ai 


aircraft. 
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simulator, the VMS became quite popular for rotorcraft 
research and Space Shuttle approach and landing investi- 
gations. Regarding the Shuttle, the landing gear and 
ground reactions were simulated to such a degree that, for 


example, the effects of blown tires, of runway surface 
(landing on a lake bed versus concrete runway), of anti- 
skid system design changes, and of nose-wheel steering 
could be studied. This was also the first use of the VMS 
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Figure 4. Vertifical motion simulator. 
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as a training tool. The Shuttle folks have continued to use 
the VMS about twice a year, six weeks each entry, to 
cycle through all the pilot-astronauts in a combined 
systems-development and training activity. 

Getting back to the subject of rotorcraft simulation, 
the VMS was also used, quite successfully, in the devel- 
opment of helicopter IFR certification criteria, in the 
development of Army Light Helicopter design specifica- 
tions, and in Army helicopter accident investigations. It is 
currently being used to investigate Civil Tilt Rotor 
approach criteria and how these are affected by various 
levels of control and display sophistication and winds. 

Also during the 1980s, a new simulation capability 
was established expanding further our FAA relationship. 
This was the introduction of the Man-Vehicle Systems 
Research Facility, a simulation facility with two transport 
cockpits (fig. 5). This facility provides very high fidelity 
representation of total missions and is used for studying 
the human factors issues in the aviation system. I would 
say that about two-thirds of all the work that is done in 
that facility is done jointly with the FAA. 

In the 1988-89 period we developed, with the Army 
as partner and CAE as contractor, the Crew Systems R&D 
Facility (fig. 6) to address helicopter crew-station design 
issues — driven in the near term by the one-versus-two- 
crew LHX issue. This simulator is also a full-mission 
simulator, which allows the flying of complete missions 
as a member of a scout attack helicopter team with all the 
threats represented. That is a veiy significant capability. 
Another special feature is its visual display capability, 
with its virtually unlimited field of view provided by a 
helmet-mounted display. It is a very impressive system. 


Over the years, simulator visual displays have been 
significantly improved and been made increasingly com- 
pelling. The effects of disharmony between visual cues 
and motion cues on the human body, factors in simulator 
sickness, become increasingly apparent. The simulators I 
talked about earlier are being used in a joint research pro- 
gram designed to shed more light on that particular 
subject. 

A final topic I would like to discuss is research 
directed at the human factors issues associated with the 
use of pilot night-vision devices. Apparently, both the 
Army and the FAA are interested in this topic. Civil oper- 
ators have asked for certification to enable them to use 
such devices in various aspects of their civil missions. As 
a result, research is being conducted in the simulators at 
Ames and in the Cobra helicopter (fig. 7) to address these 
issues. 

In summary, we have seen major changes in simula- 
tion technology and in the way simulators are used. Those 
of you in the commercial simulator business have seen an 
enormous number of changes and have implemented a 
number of very significant technological advances over 
the years. During this period Ames and the FAA have 
enjoyed an excellent relationship, one in which rotorcraft 
simulation has played an increasing role. 

As a result, we are certainly pleased to cohost this 
Workshop with the FAA. I want to wish all of you a very 
productive meeting and a pleasant stay in the Bay Area. I 
hope that later in the week you will avail yourselves of the 
opportunity to visit Ames Research Center where you can 
see some of the hardware I have spoken of this morning. 
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Figure 6. Crew Station Research and Development Facility. 





Figure 7. TH- IS Cobra. 
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C. Thomas Snyder has been Director of Aerospace Systems at NASA Ames 
Research Center, Moffett Field, California, since 1985. He is responsible for a broad 
program of research and technology development in the areas of advanced aircraft 
concepts and systems, human-machine system integration, and automated systems. 
Mr. Snyder also has operational responsibility for the National Full-Scale Aerody- 
namics Complex and major simulation facilities. He has a master of science degree 
in aeronautics and astronautics and the degree of Engineer in Aeronautics and Astro- 
nautics from Stanford University. Mr. Snyder is a former member of the Board of 
Directors of the American Helicopter Society, and was the 1986 recipient of NASA’s 
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KEYNOTE ADDRESS 


JAMES D. ERICKSON 


I must admit I am new at this business of keynote 
addressing, but when I was asked to speak at this work- 
shop I gladly accepted. I like to talk about things that I 
have strong personal feelings about, and simulation is cer- 
tainly a subject that qualifies. 

I always like coming out here to Ames. I like to see 
the latest and greatest in tomorrow’s technology, and I 
like to see advanced hardware, hardware that really flies 
and really performs. I like being in and around the R&D 
community. It is always interesting, it is always inspiring, 
and it is always exciting to be with R&D people and to 
hear people like Tom Snyder tell us what state the tech- 
nology is in. I know a lot of you work in R&D-related 
jobs as well. 

One of the things that I learned is that if you take 
material from only one source, it’s called plagiarism, but 
if you take it from several sources, it’s research. I have 
learned, too, that without management support you cannot 
implement programs that make all the sense in the world 
and that with management support you can implement 
programs that make no sense at all. And that is not at all 
unique to the R&D community. 

I want to concentrate on three things while I am here: 

(1) how far the business of rotor simulation has come, 

(2) what are some reasons why it is not where the trans- 
port airplane simulation is, at least in terms of use, and 

(3) what are we trying to accomplish here. I think some- 
times when you are frustrated by the inability to make 
progress as fast as you think you should, it is particularly 
useful to reflect back on what has been accomplished. 

I remember my first exposure to simulators. It was in 
1961, a fixed-wing aircraft, fixed-based T-37 simulator. 
Thirty years ago! I thought it was the neatest thing I had 
ever seen. I had had a couple of rides in the aircraft and I 
thought this thing was magic, it was so real. And believe 
me, a fixed-base simulator can be real. I don’t know how 
much motor sensing is provided by the eyes; I am sure 
some of you here can tell me that, but I think it is a very 
high percentage. And let me tell you about a personal 


experience. It’s a true story, a little story about Jack 
Cayot. Jack is a past manager of the FAA office here at 
Ames. He managed the office for many, many years. He 
was a person with a flight-test background like me and Ed 
Boothe. I happened to be out here flying on a simulator 
program several years ago and Jack was so excited, 
because NASA had the first daylight four-tube visual 
display that had been developed for rotorcraft. There were 
three tubes across the front, one in your direct field of 
view, one to give you a little more perception forward, the 
side one for lateral sensing. But the fourth tube was the 
real key, it was focused downward to provide contact with 
the surface, something that rotorcraft pilots can under- 
stand. Revolutionary stuff back then. 

Unfortunately, it had not been put on a motion simu- 
lator yet, and it was sitting on the floor of a very large 
storage room downstairs in the simulator building. Jack 
kept apologizing because it was a fixed-based system. He 
kept saying, “I wish we would let you see it on a motion 
system.” Well, when we got down to the simulator, I got 
in. I got the thing into the air, manipulated the controls for 
a while, and made a couple of patterns around the airfield 
that they had there. Even though the pictures were kind of 
cartoonish, I was amazed at how much I felt I was really 
in a helicopter. 

One test of my burgeoning pilot skills at that point 
was to do sideward flight. I positioned myself in front of a 
row of hangars and started doing sideward flight, faster 
and faster toward the left. I was right in front of a row of 
hangars and there was a lot of detail on those hangars; you 
could see knobs and doors and windows, that sort of 
thing. 

The four-tube visual display took a lot of computer 
capacity in terms of the computers of that era. It so hap- 
pened that the control drivers for the control system for 
the simulator also came from the same computer network. 
Things were flashing by the window at a pretty good clip, 
and if I had known a little bit more about computers I 
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would have surmised this was eating up a whole lot of 
computer capacity, but I didn’t. 

When it came time to begin slowing down there was 
no response. I started moving the controls toward the 
hover position, back to the right. But things just kept 
progressing faster and faster to the left. Soon the controls 
were against the full right stop and still we went faster. I 
was okay, because every once in a while I looked around 
the room, looked at the concrete floor and at all the junk 
piled around the room. Jack was standing directly behind 
me. He was holding onto the seat, looking over the seat to 
coach me through this new bit of technology. When I 
looked back to show everyone that the stick was full right 
and that we were still moving faster and faster to the left, I 
saw a panic-stricken Jack Cayot holding onto the back of 
the seat with terror on his face, genuine terror, and stand- 
ing in a body position preparing for a crash. It looked kind 
of silly. Here I was looking back at a man with years of 
flying and testing experience standing on a concrete floor 
in a room piled full of junk preparing to crash. I will never 
forget it. It was powerful evidence to me of the very great 
power of visual systems. 

Getting back to the T-37 simulator I flew 30 years 
ago, I thought it flew remarkably well. The technology 
existed back then, minus the visual systems to simulate 
instrument control motions and noise so that you thought 
you were flying the real aircraft. There were vacuum 
tubes and big rooms were needed to hold them, but the 
basic technology was the same. 

The other day a pilot said to me, “You know, these 
helicopters are starting to fly like simulators.’’ These heli- 
copters are starting to fly like simulators. That says to me 
that we have come a long way. I don’t know if that man 
was saying that artificial control systems are making air- 
craft sort of feel artificial, or that the simulators are just 
getting better and getting more like the aircraft. I failed to 
ask him. But simulators are now able to fly very much 
like the aircraft. Why, we are even effectively using simu- 
lators to design an aircraft’s control system before the air- 
craft ever flies. Who could have thought 30 years ago that 
we could be doing that? 

But I would argue that today’s simulators are valu- 
able tools even if they did not fly like the aircraft. It is 
important that you know the value of what you have. Like 
the story of two ladies walking along a Fort Worth side- 
walk. This is a true story. I am from Fort Worth. They are 
walking along when a frog jumps out in front of them. 
They tried to get around the creature but he said, “Don’t 
pass me by. Kiss me and I will turn into a Texas oil man.” 


One of the ladies picked him up, put him in her purse and 
closed the purse. The other young lady said, “Aren’t you 
going to kiss him?” “Heck no,” the other one said, “a talk- 
ing frog is worth a lot more than a Texas oil man.” 

So you have to recognize the value of what you have. 
Several pilots have said that FlightSafety’s 222 simulator 
does not fly like the aircraft, and I guess I have con- 
tributed a few comments like that myself. That does not 
mean the simulator is not a very valuable training tool, or 
that it is not a valuable simulator. The simulators today 
have a wonderful capability to not only simulate but to 
surpass or to extend what is possible in the aircraft. Let 
me explain. 

There are diabolical training scenarios known to 
rotorcraft pilots who have flown simulators that cannot be 
duplicated anywhere else. Things like critical instrument 
failures, high-side governor failures on twins, twin-rotor 
failures, progressive engine and transmission failure, 
those kinds of things. Failures like these cannot be set up 
with any degree of credibility and safety in the aircraft. 

But a simulator can actually give the pilot the opportunity 
to experience something very close to real-world symp- 
toms and real-world conditions, and to train the real-world 
motor skills necessary to deal with such problems should 
they occur in the aircraft. With today’s simulators you can 
equip the pilot to recognize and deal with symptoms that 
he or she would otherwise see in the aircraft for the first 
time only under actual emergency conditions. What a 
marvelous tool. You can give crews the experience base 
to deal with these situations before they ever happen in 
the real world. How many of you are rotorcraft pilots? Oh, 
my goodness. You could have a pilot’s convention here. 

In my experience there is nothing quite like a high- 
side governor failure in a twin-engine rotorcraft. For those 
of you who are not pilots, let me explain. In a twin-engine 
rotorcraft, the two engines share the job of powering the 
rotor. If a high-side governor fails, one of the engines 
loses its governing capability and begins to put in excess 
power. When the good engine senses that the other engine 
is overspeeding the rotor, it begins to decrease its torque 
and power. Now, if the pilot isn’t paying close attention, 
the initial symptoms can cause him to think there has been 
an engine failure. As a result, he will treat the good engine 
instead of treating the engine that has had the failure. 
Experiencing such a failure in a simulator, and talking it 
over with the crew and with the instructor, can prepare a 
pilot for the real thing, even for something as subtle as a 
high-side governor failure. Simulators are great tools. I 
tlnnk we all agree v . 1th that. But I probably need to add 
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that some rotorcraft simulators do fly just like the real air- 
craft. I had a testimonial to that from Jack Hart this morn- 
ing about one of those simulators and how good the 
fidelity really is. But back to one of the issues I promised 
to talk about: Why hasn’t rotorcraft simulation progressed 
to the same level of development and use as transport 
airplane simulation? 

I think there are a couple of simple answers to that 
question. Up until 15 years ago, an IFR flight in a civil 
helicopter was almost unheard of. Sure, the military was 
doing it, because you don’t prepare to fight a war only on 
clear days. Even in the military, however, IFR was the 
exception, not the rule. So the commitment to simulation 
of rotary-wing flight came much later than it did in the 
commercial airlines where IFR for every aircraft and for 
every crew was a necessary condition for doing everyday 
business. 

The fixed-wing pilots were taught IFR flying at an 
early stage, particularly in the military. Those who transi- 
tioned to rotorcraft brought IFR skills with them. But 
today there are many civil helicopter pilots who do not 
have instrument ratings. Many of the civil missions are 
utility VFR applications for which an IFR simulator has 
only limited value. Thus, the late start in rotorcraft simula- 
tion: the lack of a mission that demanded IFR capabilities 
on every flight, particularly on the civil side. 

I think we have to remember that the fixed-wing 
experience is out there as a benchmark for us and, need- 
less to say, we have to be alert so that lessons learned in 
transport airplane instrumentation in use of simulators are 
not repeated. The technology is on hand. Pilots report that 
the XV- 15, the S-76, and V-22 all have excellent fidelity. 
It’ s not a matter of mastering the technology to make the 
devices fly like real aircraft. I said earlier that we were 
pretty much on our way with the T-37 simulators many 
years ago. That technology was brought forward very 
effectively by the military programs and by all the mili- 
tary pilots who were trained with those marvelous 
machines many, many years ago. The job has been han- 
dled well in terms of technical development, but the tech- 
nology has not been able to master and reduce the cost of 
simulation. 


I believe there are great opportunities to lower the 
cost of simulation. I don’t have the answers, but I do 
know that if Jack Cayot could be convinced he was about 
to crash while standing on a concrete floor in a store 
room, there are possibilities for decreasing the cost of 
motion systems. I do know that as long as simulators cost 
more than the aircraft they are simulating, there will be an 
economic disincentive to simulation. I know, too, that we 
are making wonderful advances in every area of electrical 
and digital technology so that there are opportunities on 
the horizon for reducing the cost of everything that the 
pilot sees in simulation. I know that there is a lot we can 
do, and that what we can do in this area is inherently good 
for the advancement of the state of the art of rotorcraft 
simulation. And it will lead us to a point where everyone 
can afford to send every pilot through simulator training 
on a regular basis. There is a challenge and an opportunity 
here. There is a challenge that I would make to each and 
every one of you: when talking technology over the next 
three days, include the word “cost” somewhere in your 
thoughts. I am not sure we always do that. And it is my 
opinion that that is where many of the opportunities lie. 
We have come a very long way in mastering the technol- 
ogy and in articulating the standards for design. The 
opportunity is in mastering the cost of those technologies 
and managing the costs imposed by the standards that we 
require. 

I would like to say just a couple of words in support 
of Ed Boothe’s public meeting, which I understand is 
going to be on Thursday. We in the FAA really seriously 
need your thoughts and your best words and your wisdom 
on that activity. It is important that we in government not 
make decisions in the dark. When we do, they are inher- 
ently bad decisions. Please come to that meeting prepared, 
and please express your thoughts in the meeting. The 
FAA is counting on you. 

I hope each of you has an exciting and productive 
conference. In glancing over the agenda this morning I 
saw a whole variety of interesting subjects dealing with 
people, technology, theory, equipment, and standards. I 
am anxious to hear the presentations and I look forward to 
meeting many of you while I’m here. 
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1. HELICOPTER SIMULATOR STANDARDS 

EDWARD M. BOOTHE 


I will reiterate some of the things that Jim Erickson 
said, but my main purpose is to discuss the work that has 
been done on the Helicopter Simulator Advisory Circular, 
120-XX. 

First I would like to thank all of you for being here 
and for supporting this activity. I know it is quite an effort 
for you, and not without expense. But, as Jim said, we in 
the regulatory business certainly need your support and 
input; in fact, we can’t do our job without it. 

I would also like to recognize that we have a good 
deal of international support. We have friends and repre- 
sentatives here today from New Zealand, the United 
Kingdom, Canada, and France. That is a fine representa- 
tion, and it is appreciated. If I failed to mention a country 
that is represented it is only because I didn’t meet that 
representative this morning. 

I want to briefly describe where we are on helicopter 
simulator standards. As Jim said, the advantages of simu- 
lation have long been enjoyed by the airplane community, 
and the use of simulators has expanded steadily. Since 
1980 there has been an average annual increase of 14% in 
the inventory of airplane simulators used by the entire 
U.S. Air carrier and corporate aviation industry. I think 
that is pretty remarkable: there were about 88 simulators 
in 1980, which were, by today’s standards, not very 
sophisticated, and just a month or so ago we exceeded 
300 simulators that are in service to U.S. industry. But 
that capability has not been available to the civil heli- 
copter community. I know a good deal of simulation 
capability has been available to the military services but 
not to the civilian community. 

The need has been pointed out in previous meetings 
and workshops, and I think it is becoming obvious. In fact 
I just read on the way out here that some people are refer- 
ring to aircraft as part-task trainers. And I think that is 
true. As Jim mentioned, there are so many things that one 
can do in simulator training and checking that simply 
cannot be done in aircraft. At a recent meeting, a paper 
from Norway described an incident in a Super Puma. 


There was a tail-rotor failure at hover, but the crew recog- 
nized the failure immediately, recovered with no damage 
to the aircraft and no injuries to the crew. Complete credit 
for that quick failure recognition and quick recovery was 
given to the crew’s practice of that precise failure in the 
simulator. It is this kind of event that causes the aircraft to 
be called a part-task trainer. You cannot do the whole job 
of training in the aircraft. Moreover, there is a big and 
favorable cost factor involves ;,i simulator training. 

But getting on with this, the history of trying to estab- 
lish some civil standards for helicopter simulators goes 
back at least to the meeting we had in Atlanta in 1984, at 
which time we had a fairly general review of the state of 
the technology. The following year we had a working 
group that did produce a draft advisory circular for heli- 
copter simulator standards, but it never progressed. One of 
the reasons, I think, was because the federal regulations 
that control training and checking for airmen do not rec- 
ognize any credits for helicopter simulators. 

Then just last year the Royal Aeronautical Society 
had a seminar on helicopter simulation and again the 
interest and the need were indicated and the use of simula- 
tion at that time in North Sea oil operations was pointed 
out. Of course that is more of interest, you might say, to 
the United Kingdom and, in this case, Norway, where a 
number of simulators are in use. But there are only two 
civilian helicopter simulators in use in the United States, 
and I think that is because the FAA permitted use of heli- 
copter simulators for pilot certification only by exemp- 
tion. So there is no general credit. Consequently there are 
no current standards, which is why we are here. 

Those two simulators, although being limited in their 
applications, still are quite valuable in their use. The 
Bell 222 has no hover credits, and it was qualified by 
using an old interim standard that we were working on. 
What we really did was have four expert pilots (I think 
Jim Erickson might have been one of them) who flew the 
aircraft for a few minutes and then performed the same 
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task in the simulator. They then went back to the aircraft 
and then back to the simulator. 

Those of you who have in the past been involved in 
handling-qualities work understand, I am sure, that after 
20 minutes you might as well get out of that device and 
into the next one since pilot adaptation time is usually 
quite short. 

The next one was the S-76. At that time we did have 
the interim draft standard that was produced in 1985. But 
because nobody expected that, there really were no data 
for the aircraft, at least not to the extent that we needed 
them, so we used what data were available. We did the 
same routine that I just mentioned with some expert 
pilots, and we qualified the simulator and developed an 
exemption through a petition from FlightSafety for credits 
for that simulator. And in fact, you can do most of an ATP 
certification check in that helicopter simulator with only 
about three or four follow-up maneuvers for validation in 
the aircraft. So we know it can be done. 

So why try again now? We still only have two simu- 
lators. I think in the last five years we have certainly 
increased our knowledge and our experience. Some of the 
questions we had about standards five or six years ago, we 
now have answers to. One small example is control load- 
ing. Six years ago when we said you have to simulate the 
break-out forces in a helicopter control system, most peo- 
ple in the business said we couldn’t do that. But I know 
for a fact that today we can do that and that we can do it 
quite well. And it will stay constant, not changing as each 
person uses the machine. 

I think here today we are going to follow up on these 
past issues. But another very important thing is happen- 
ing: the FARs are being revised. There is a new draft 
Part 142 that primarily addresses training centers. The 
notice of proposed rule-making for that effort will be 
available this summer with, we hope, a rule by some time 
in late 1992. It will permit training and checking credits 
for helicopter simulators. I think that is primarily going to 
be started at the higher levels of pilot certification. But at 
least that is really where we are in airplane simulation. 
There are not many simulator credits for the lower levels 
of pilot certification; they are all pretty much at the upper 
levels. 

One objective of our efforts this week is to form a 
working group of experts who will meet as necessary to 
address these issues and to establish standards and guid- 
ance. This process has worked exceptionally well, 
although slowly, with airplane simulators. And the devel- 
opment process for airplane simulator standards is cer- 


tainly applicable to helicopter simulators. Over the past 
15 years the standards for helicopter simulators have pro- 
gressed such that they are almost as remarkable as the 
technology, but the idea has been to keep them in step 
with the technology. 

In 1978 we did the first crude landing approval in a 
simulator. And now we are doing total pilot training and 
checking in simulators, and the standards have been 
revised to reflect that. The working group process has 
worked; in fact it worked to the extent on the lastest air- 
plane standard that that standard has been accepted as the 
core of international standards for airplane simulators. We 
hope that will become an International Civil Aviation 
Organization (ICAO) policy or handbook for international 
use for commonality and qualification of airplane simula- 
tors. The point is the process should be equally applied to 
helicopter simulators. 

It is hoped that the working group membership we 
seek will represent a broad range of the community of air- 
craft and simulator manufacturers, users, and operators. 
And, of course, the final customers, the training experts, 
the technical societies, and the regulatory authorities must 
be represented. We would like the group to be limited to 
about 30 members; our experience shows that with more 
than that, it is very difficult to make progress. In fact, on 
the international working group, Brian Hampson, who is 
the chairman for the Royal Aeronautical Society, has 
made a special effort to limit the size of the group. I thank 
him for that, and I think a great deal of the progress that 
has been made is a result of keeping the same members 
meeting after meeting and because we have limited the 
group to those same members. Even so, we still rehashed 
a lot of stuff. 

As Harry Reasoner once put it, helicopters are differ- 
ent. Some pilot tasks are more demanding in the heli- 
copter simulator than they are in helicopters. We have 
noticed that the hover and low-speed tasks have been the 
most challenging to simulate. That is one reason the 
Bell 222 is not qualified for that, although it probably 
could be with some updates. Progress was made in that 
area, however, so that the S-76 is so qualified. Not all 
pilots agree that that should be true, by the way, but that is 
the nature of these kinds of activities, I think. 

Not all simulators need to be qualified for all tasks, so 
we will be looking at a number of levels of simulators. 

We have tried to keep those levels aligned with what has 
been successful for airplanes, mainly so we can keep the 
record straight. And v"> oe working later in this 
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Workshop to form the group that will follow through with 
this effort. 

So, if you would be kind enough, then, please review 
the draft document. It is modeled, in terms of general 
policy and structure, on the airplane document. But that is 
a matter of style, not content. And we would like to 


pursue that approach because we have spent years actually 
finalizing that format and structure. Nevertheless, the 
technical content is certainly something that needs to be 
addressed, and addressed in fine detail. I will look for- 
ward to hearing from you on Thursday when we form the 
standards working group. Thank you. 



Edward M. Boothe is manager of the FAA National Simulator Evaluation Program, 
Atlanta, Georgia. He is responsible for ensuring that all simulators used in checking 
U. S. civilian airmen meet appropriate standards. Before joining the FAA, he was a 
research engineer and engineering pilot at Calspan Corporation, where he performed 
airplane handling-qualities research and control-systems studies. Mr. Boothe has a 
masters of science degree in aerospace engineering from Texas A&M University, 
and has an Airline Transport Pilot Certificate with ratings in Boeing 757/767 air- 
planes. He serves on the AIAA Flight Simulation Technology Committee, and is an 
Associate Fellow of the AIAA and a Fellow of the Royal Aeronautical Society. 
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2. HELICOPTER SIMULATION: AN AIRCREW TRAINING AND 
QUALIFICATION PERSPECTIVE 


RICHARD A. B1RNBACH AND THOMAS M. LONGRIDGE 


FAA goals for the training and qualification of com- 
mercial aviation rotary-wing airmen are no different from 
those in the fixed-wing categories — to improve safety 
through effective training and checking. Flight simulators 
have been successfully employed for this purpose in the 
air carrier community for a number of years, and the FAA 
has developed an explicit set of regulatory compliance 
requirements in that regard. The recently established 
Advanced Qualification Program (AQP) expands the 
regulatory boundaries for device-based fixed-wing train- 
ing and aircrew qualification, by allowing for families of 
devices lower on the equipment complexity continuum 
than the traditional categories of flight simulators. 
Although our understanding of the issues involved in 
qualifying synthetic devices for such applications is 
becoming increasingly mature, this circumstance is decid- 
edly not yet the case for rotary-wing application. We wish 
to review some of the unique considerations which 
( 1) distinguish the commercial rotary-wing domain from 
its fixed-wing counterpart, and (2) motivate the FAA to 
proceed cautiously in extrapolating from our fixed-wing 
experience in establishing qualification requirements for 
helicopter simulators. It is proposed that the issue of 
device qualification should be considered in the context of 
an overall training and qualification system. Rather than 
focusing solely on the isomorphism between the engi- 
neering characteristics of the synthetic device versus the 
aircraft, such an approach would integrate engineering 
and behavioral criteria. Ideally, a decision strategy on 
helicopter simulator fidelity requirements would include 
consideration of the proficiency objectives on which air- 
men would be trained and qualified using the device. 

Good afternoon ladies and gentlemen. I’m honored to 
have an opportunity to share my views, and more impor- 
tantly the views of the Federal Aviation Administration, 
of our regulatory goals for the use of helicopter flight 
simulators and helicopter flight-training devices. 


Although I may spend a lot of time and energy high- 
lighting the differences in helicopter and airplane require- 
ments later in this presentation, I am going to start by 
saying that the FAA’s regulatory goals for flight simu- 
lation are exactly the same for helicopters as they are for 
airplanes. These goals are to increase safety in flight oper- 
ations, to ensure attainment of reasonable aircrew profi- 
ciency standards, and, through better trained crews, to 
foster the safe and efficient growth of the aviation 
industry. The FAA recognizes that flight simulation is a 
proven and effective means of attaining these goals. 

The FAA considers its experience in flight simulation 
to be a positive example of how the industry and govern- 
ment can cooperate to achieve their sometimes diverse 
goals. Through foresightedness, dedication, and plain hard 
work, we, both government and industry, have made the 
use of airplane simulators one of the most successful pro- 
grams ever undertaken to increase safety and efficiency. 
Our simulation programs have been an unqualified 
success. 

Aircrews recognize and appreciate the use of flight 
simulators because of their proven ability to enhance the 
crew’s performance. The FAA, airlines, and the traveling 
public benefit immeasurably from the safety improve- 
ments simulator training has brought to day-to-day opera- 
tions. Before simulation came into widespread use, 
required airline training activities contributed substantially 
to airport congestion, delays, and noise problems, as well 
as to other environmental issues. In today’s airline 
training environment, air-traffic control doesn’t have to 
accommodate the training that is done in simulators, and 
aircraft and fuel resources are conserved. We anticipate 
even greater progress in these areas with the advent of 
increasingly sophisticated but low-cost flight-training 
devices. At the FAA, we see no reason for any lesser 
degree of success in the use of helicopter flight simulators 
and flight-training devices. Interestingly, this has not yet 
occurred. 
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Let’s take a quick look at where we are in the FAA 
with respect to helicopter flight simulation. The helicopter 
simulator has no detailed regulatory basis, such as the 
airplane simulator has in Appendix H of Part 121. The 
operating and airman certification regulations do not have 
provisions for use of helicopter simulators that parallel 
those of airplane simulators. However, we do have a draft 
helicopter simulator qualification advisory circular which 
has been used as an interim standard in approving two 
civil helicopter simulators. I participated in the evaluation 
of these simulators and would like to share my thoughts 
and observations about them with you. I believe we 
should be cautious in extrapolating from our airplane 
flight-simulator and flight-training device experience. I 
also feel that the overall training and qualifications sys- 
tems for helicopters are not directly equivalent to airplane 
training systems. 

Helicopters not only look and sound different than 
airplanes do — they have different missions and require 
different crew skills. Although helicopters can be used for 
some of the same mission tasks as airplanes, they also can 
do missions an airplane could never accomplish. Heli- 
copters are capable of operating in natural and man-made 
environments that are prohibitive to airplane operations. 
Helicopter pilots must learn how to control their aircraft 
in any possible combination of directions of flight. The 
helicopter’s mechanical and electronic equipment combi- 
nations have complexities not usually found in airplanes 
of equal size. All these factors enable the helicopter’s 
wonderful freedom of navigation. However, they also 
introduce a high potential for risk in helicopter operations 
that must be recognized and accommodated through effec- 
tive crew training. These differences have a critical influ- 
ence on the design of helicopter simulators and on the 
overall design of any helicopter crew training and qualifi- 
cation system. 

Let’s compare the issues that differentiate helicopter 
from airplane operations. In general, airplanes are used for 
transportation of persons or cargo between airports. Mis- 
sions that airplanes and helicopters share include training, 
recreational flying, crop planting and protection, pipeline 
and power-line surveillance, livestock surveys, aerial 
photography, aerial search, and surveying, as well as 
short-range transportation between airports. Helicopters 
are the primary means of air transportation between off- 
airport landing sites and are also used in construction 
work, law enforcement, emergency medical transporta- 
tion, and rescue operations. The special operations that 


helicopters can perform that airplanes cannot are too 
numerous to list. 

Helicopter crews may be called on to perform all 
these missions in the same physical environment that air- 
planes usually operate in. However, in many instances, 
helicopter missions are performed in environments not 
shared by airplanes. There are substantial differences in 
the characteristics of the many landing and surface operat- 
ing areas used by helicopters. In contrast, airplanes always 
use some form of level runway with cleared approach and 
departure paths. Except at permanent heliports and air- 
ports, helicopter crews must reconnoiter, select, and exe- 
cute every detail of the surface operation without benefit 
of airport engineering and improvement activities. In 
many cases, helicopter operating sites are not located in 
controlled airspace and have only limited support from the 
air-traffic control systems, federal navigational aids, and 
weather reporting and forecasting systems. 

In addition to dealing with a more complex operating 
environment, helicopter crews must cope with the han- 
dling characteristics of the helicopter that permit its nearly 
unrestricted mission capabilities. The very features that 
make the helicopter so versatile also increase the diffi- 
culty of its operation when compared with airplane flying. 
Airplane and helicopter flight-path management and con- 
trol characteristics are different. Airplanes can’t fly side- 
ways or backward. Helicopters, of course, can fly in any 
direction. The crew knowledge and skills required for 
sideward and rearward flight are not a consideration in 
airplane operations. 

Most airplanes share a lot of common handling quali- 
ties. For example, the basic handling qualities of a Cessna 
twin are not very different from those of a single-engine 
Beechcraft. This can’t be said for helicopters. Handling 
qualities may substantially differ from one helicopter to 
another. Compared with airplanes, helicopters are a rather 
unstable aircraft with high work loads. Airplanes are 
mechanically simple devices when compared with heli- 
copters. This increased mechanical complexity requires 
helicopter crews to learn and understand a greater number 
of abnormal and emergency procedures. Helicopter pilots 
would be quite surprised to check out in a new helicopter 
without learning how to cope with failure of anti-torque 
control. How many fixed-wing pilots have been taught 
what to do if rudder control fails? 

Each of the differences I’ve mentioned can have a 
profound effect on helicopter flight-simulator and flight- 
training-device design u a direct extrapolation of our 
perience with airplane simulators may, there, be 
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inappropriate. Let's summarize what should be accounted 
for in helicopter simulator design. 

First, let’s consider the conditions that apply to air- 
plane and helicopter simulators. Both require accurate 
simulation of aircraft system operation, IFR en route 
navigation, IFR and VFR terminal-area navigation, and 
airport surface operation. A second list applies to addi- 
tional helicopter flight-simulation device design consider- 
ations. This second list of considerations includes VFR 
en route navigation, lateral and rearward flight, offshore 
operations, water surface operations, amphibious opera- 
tions, urban congested-area operations, slopes, confined 
areas, flight-path obstructions, autorotations, and power- 
off landings. 

Let’s assume that in the near future we determine 
what helicopter simulators and training devices should be 
capable of and let’s further assume that the FAA publishes 
a final version of advisory circulars for helicopter flight 
simulators and helicopter flight training devices. What can 
we use them for? In their present state, the Federal 
Aviation Regulations, Pilot Test Standards, and other 
regulatory documents permit only very limited use of 
helicopter simulation. Therefore, when we develop cri- 
teria for helicopter simulators and training devices, we are 
only half finished with the job at hand. We need to deter- 
mine what the appropriate proficiency objectives are for 
helicopter crews and amend the FARs to enable device- 
based training and checking for those proficiency 
objectives. 

Which should we develop first, the helicopter crew 
qualification standards, helicopter flight-simulator and 
flight-training device standards, or the enabling Federal 
Aviation Regulations? Tom Longridge and I believe we 
should view these three tasks as an integrated job that 
requires development of helicopter crew qualification 
standards, helicopter flight-simulator and flight-training- 
device criteria, and development and implementation of 
changes to Federal Aviation Regulations in support of 
modern helicopter training and qualification requirements. 
We believe that we can and must take a systematic 
approach to the development of an overall training and 
qualification system, because without systematically 
developed crew qualification standards and enabling 
FARs, we have no means to ensure that we will effec- 
tively be able employ any helicopter-simulator or 
training-device criteria. 

To determine what skills helicopter pilots need to 
accomplish their job, we need to take a look at the 
mission-related tasks today’s helicopter pilot must master. 


Qualification standards for helicopter crews can be devel- 
oped and adopted for use in an integrated training and 
qualification system which is designed to include the 
flight simulator and flight-training device as essential 
tools for learning and evaluation. 

Given the environment in which helicopters operate, 
their flight characteristics, and many mission tasks, high- 
fidelity helicopter simulation is technically very challeng- 
ing. For the average commercial operator, it may in fact 
simply be too costly. For that reason recommendations on 
fidelity requirements should carefully weigh cost versus 
benefit in light of the purposes for which these devices 
will be used. 

Flight simulation, by definition, always represents 
some degree of abstraction from reality, for the simple 
reason that a simulator is not an aircraft. Therefore, there 
will always be some degree of compromise on realism. 

So, a fundamental issue is the decision criteria on which 
basis such compromises should be determined. Certainly 
engineering criteria, such as the extent to which the simu- 
lator’s display system duplicates the actual aircraft’s field 
of view, or the aeromodel duplicates the actual aircraft 
flight characteristics, are a very important consideration in 
any such decision process. However, from a training and 
qualification perspective consideration of how the device 
is to be used is of equal importance. We feel that for heli- 
copter simulators and flight-training devices, because of 
their many unique characteristics, a sensible decision 
strategy on fidelity issues must integrate both engineering 
and behavioral criteria. 

MR. TREICHEL: Regarding Part 142 in the pro- 
posed rule-making, is there some kind of advisory team or 
committee that is being made up that some of us could get 
involved in to make sure that everything is running along 
as smoothly as this effort is? 

MR. BIRNBACH: During one of the breaks I am 
going to introduce you to Warren Robbins who is here 
with us from the General Aviation Division. Part 142 is 
the product of an advisory committee. It was not quite an 
advisory committee when they put it together, so I would 
rather not talk about it to any great extent, but it included 
people from the simulation industry and from the training 
centers and the helicopter industry. And it is not a bad 
document. But I will get you together with Warren and 
you can talk directly about it. Anybody else? Yes, sir. 

MR. RUTKOWSKI: You only have two simulators 
approved right now. What is the requirement . . . how 
many other operators out there do you have with the need 
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for that kind of fidelity? How big is the need out there to 
build this type of device? 

MR. BIRNBACH: I cannot answer that for the 
Part 6 1 operators except for one thing. I know that what 
we call a Part 91 operator has a little problem in exposing 
the assets that they have. If someone owns an S-76, a 
Bell 222, or an SA-360 type of machine, it is really tough 
to go out and ask them to do tail-rotor failures and touch- 
down autorotation in these things. The insurance company 
knows it and FlightSafety’s Greg McGowan can tell you. 
The real problem is the industrial-type operator, the off- 
shore operator, the air taxi, the external load operator. 
These people have a little difficulty with what simulation 
is available to them and they cannot do the kind of tasks 
they need to do for their pilots. So it is difficult to answer 
your question from my perspective. There is not a lot of 
demand right now in the 135 world for helicopter 
simulators. 

MR. RANDALL: Over the last 30 years I haven’t 
seen a lot going on in behavioral science things. I think it 
is desperately needed when we transition into helicopter 
simulators. 

MR. BIRNBACH: Let me try to answer that as best I 
can. First, I don’t want to throw the baby out with the bath 
water. I don’t want to restart this whole issue of what 
should come first and what should come second. With 
respect to the level of helicopter simulation available to 
us, that would be covered by the draft advisory circular. I 
think we are smart to go ahead with that right now, and 
the rule-making projects that we have in hand will support 
the use of those types of simulators. Where we really need 
to make sure we do this is in regard to part-task trainers or 
training devices. It is going to be very important to us, 
especially in rotary-wing, but just a little bit less so in 
fixed-wing training devices. How do we give part-task 
credit? Last year the FAA came out with an integrated 
human factors program. We came up with a plan which is 
in the final approval stage. In that plan are work resumes 


and intents to go out and do research on these issues. We 
need to do some research, we need to come up with the 
processes for giving credit for part-task devices. Then we 
need to do something about clarifying the rules. I do not 
see that happening in the next 6 months, but I see the first 
steps being taken to do it. 

MR. WALKER: We have been dealing with heli- 
copter simulator operations, and one of the issues that’s 
been of most concern to me is in your decision criteria. In 
particular, 1 always see a problem with having part-task 
data that are tailored to support simulator development. Is 
the regulation that you are addressing going to deal with 
this issue? 

MR. BIRNBACH: We have talked about these 
things between Ed Booth’s shop and mine and some 
others, on several occasions where you talk about flight- 
test data to support simulator development. And there are 
two issues here. One is to technically assimilate a flight 
training device by being able to measure what it looks 
like, what is sounds like, and what it does. 

The other is to figure out what credit you can give to 
the training requirement. There is no doubt in my mind 
that the high end of those engineering criteria is extremely 
important and that we have had success in simulator qual- 
ification relying on this. 

I do not know what to do with this decision point that 
we talked about here, and looking at how we use this 
engineering criteria as opposed to transfer of skills crite- 
ria, is when we get down into the lower-order devices. I 
just do not know how to do it. We have some people who 
have a lot of good ideas on how to determine what to do, 
but until we do that I think we are going to have to rely on 
some of our successes. We just cannot argue with the 
success that we have had in fixed-wing simulation and in 
these two rotary-wing simulators in relying on flight test 
data as our beginning point. I do not know what else to 
say to you there. 



Richard A. Birnbach is manager of the Air Carrier Training Branch at FAA head- 
quarters in Washington, D.C. He is responsible for FAA policies, procedures, and 
regulations on airline training and qualification for aircrew personnel. Mr. Birnbach 
served as pilot in the U.S. Army, where he earned ratings in both rotary- and fixed- 
wing aircraft. Following combat duty in Viet Nam, he was an instructor pilot at the 
Army Aviation Center, Fort Rucker, Alabama. Mr. Birnbach is the author of the 
FAA’s Air Carrier Operations Inspector’s Handbook. 
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3. ROTORCRAFT MASTER PLAN 

PETER V. HWOSCHINKSY 


I would like to talk about the vertical flight program 
and give you some insight into the bigger picture. Jim 
Erickson mentioned why simulation use for rotorcraft is 
not at the same stage as it is for scheduled airlines. We are 
working toward the day when the term “scheduled airline” 
includes rotor-borne flight as well. I would like to speak 
of our planning efforts that we hope will help make this 
happen. Maybe we can prove to Ed Boothe that we can 
get there from here. 

In 1975 the Rotorcraft Task Force (ROTAF) was cre- 
ated to address issues associated with industry growth and 
to provide a forum for communication between govern- 
ment and industry. As a result of that task force’s recom- 
mendations, the first rotorcraft master plan was published 
in 1983, updated annually through 1987, and again pub- 
lished in November of 1990 after extensive rewriting and 
reformatting. Although the master plan contains a com- 
prehensive summary of vertical flight goals, it alone is not 
sufficient for tracking project status and monitoring 
progress; the Vertical Flight Program Plan (VFPP) will 
provide that capability. The FAA Executive Board rec- 
ommended establishment of a vertical flight program 
focal point and preparation of the VFPP to tie together all 
vertical flight activities. 

The Board also stipulated that the plan should be 
consistent and that the policy direction from the FAA 
must be ready to ensure a hospitable environment when 
industry presents a feasible vertical flight initiative. The 
Board agreed that the program should proceed in two 
phases, with the initial version of the VFPP covering the 
Phase 1 time frame. 

Congress has shown interest in the potential that 
vertical-flight technology may hold for helping to solve 
some of the nation’s problems, especially transportation 
problems. Hearings on the civil tilt-rotor were held in 
1987 and 1990 by the House Transportation, Aviation, 
and Materials Subcommittee. In 1989 and 1990 both the 
House and Senate Armed Services Committees held 
hearings on the V-22 at which the Department of Defense 


was requested to provide a report on civil applications for 
the aircraft. 

In development of the Reconciliation Act of 1990, 
Congress requested a blueprint for additional research 
needed to develop an economically feasible civil tilt-rotor 
aircraft. The study would also identify and describe the 
types and numbers of facilities needed to sustain an eco- 
nomically feasible tilt-rotor fleet and would specify 
changes in ATC procedures that must occur if the benefits 
of the tilt-rotor aircraft are to be realized. 

Proof of further congressional interest is the Mag 
Lev/Tilt-Rotor Study currently being conducted by the 
Office of Technology Assessment. The Administration’s 
national aeronautical R&D goals include an action plan to 
enhance the safety and capacity of the National Airspace 
System through advanced automation, electronics tech- 
nology, and new vehicles concepts, including vertical and 
short takeoff and landing aircraft. In Moving America, the 
emergence of new technology such as the civil tilt-rotor is 
emphasized for its potential to provide transportation in 
dense corridors. The Office of the Secretary has requested 
that analysis be conducted into feasible alternatives. These 
studies are ongoing today. The civil tilt-rotor is considered 
a practical alternative for dense-corridor passenger trans- 
portation. Finally, the Administration has approved the 
development of a joint FAA/Industry Rotorcraft Master 
Plan. 

State and local governments have shown great inter- 
est in the tilt-rotor as a mode of transportation that may 
reduce airport congestion and provide considerable time 
savings. To date, $3 million has been awarded to various 
states and cities, and to the Port Authority of New York 
and New Jersey for tilt-rotor feasibility studies and verti- 
port studies to investigate a potential intercity transporta- 
tion system. 

The hierarchy of plans that will be used to develop 
the VFPP is based on the National Transportation Policy 
endorsed by the secretary of transportation and the FAA’s 
own National Aviation Policy for developing the air 
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transportation system through the next century. The three 
capital plans which support those established policies 
include the Capital Investment Plan (CIP), Research 
Engineering and Development Plan (RE&D), and the 
National Plan for Integrated Airport Systems (NPIAS). 

The next level in the hierarchy is represented by two 
plans that are organized along functional lines, the Avia- 
tion System Capacity Plan, and the Rotorcraft Master Plan 
(RMP). In other words, there are these cross-cutting plans 
which may contain projects that receive their support from 
each of the capital plans in the previous tier, while at the 
same time providing for funding contained in these capital 
plans. 

The levels below the RMP contain the two special- 
ized documents that will relate specifically to vertical 
flight: the VFPP and project implementation plans (PIPs); 
and Contractual Flight Program Plan and PIPs. Not all of 
the projects in the VFPP will warrant a PIP, only those 
involving a large degree of intra-agency and interagency 
coordination and effort. The VFPP will integrate projects 
from two other primary vertical flight documents, the 
RMP and the National Civil Tilt-Rotor Initiative (NCTRI) 
implementation plan. This process will eliminate 
unneeded overlaps and gaps and provide cross-plan 
coordination. 

The RMP coordinates existing programs and new 
actions needed for vertical-flight aircraft to reach their full 
potential within the NAS. Strategies and projects to 
accomplish vertical flight goals are divided into three 
issue areas: (1) infrastructure, including heliport and ver- 
tiport development; (2) NAS integration aircraft technol- 
ogy; and (3) pilot training and certification. Successful 
implementation of the RMP depends on the joint com- 
mitment of federal, state, and local government agencies 
and industry. Checkpoints described in the RMP provide 
the initial basis for ensuring that this common commit- 
ment exists at major investment decision points. The RMP 
appendix summarizes FAA and industry activities. 

In 1988 the FAA initiated a comprehensive review of 
the 1987 version of the RMP. The review involved coop- 
eration between the FAA and representatives of the 
rotorcraft industry. Efforts were refocused to emphasize 
NAS capacity enhancement using vertical flight. Integra- 
tion of a civil tilt-rotor into the nation’s air transportation 
system was a key element of the revised plan’s strategy 
for accomplishing that goal. The revised version of the 
RMP was published in November 1990. 

Vertical-flight technology has the potential to 
enhance NAS capacity at a fraction of the investment that 


would be necessary to build new or improved commercial 
airports. This potential is the underlying reason for the ini- 
tiatives presented in the RMP. The RMP will be imple- 
mented incrementally, with checkpoints existing at the 
end of each phase to measure how the system is perform- 
ing relative to the plan’s goal. Resource commitments will 
be made on a quid pro quo basis with this plan being used 
to provide justification for committing resources to high- 
priority rotorcraft projects. By 2010 rotorcraft could pro- 
vide as much as 10% of the intercity passenger operations 
capacity in the NAS. That would mean that rotorcraft 
would then account for 5 million of 50 million annual 
operations, and for 105 million of more than 1 billion 
enplaned passengers. 

As mentioned earlier, implementation of the plan is 
divided into phases, with a major investment decision 
needed at the end of each. Between now and 1996 a suc- 
cessful demonstration of the civil tilt-rotor would be 
accomplished, along with development of one or more 
heliport/vertiport networks. Between 1997 and 2000 the 
focus would be on the transitioning of vertical flight 
activities more to the private sector, with the FAA provid- 
ing technical assistance as appropriate. 

After 2000 and beyond 2010 the FAA would hand off 
responsibility for most vertical flight activities to industry, 
as scheduled passenger service matures and expands. The 
RMP implementation phases (fig. 1) illustrate the rela- 
tionship between the rate of investment of federal 
resources and the corresponding operations growth. As 
shown, there is about a five year lag between the neces- 
sary investment and the time that operations growth 
becomes evident. This time line shows the checkpoints in 
the RMP that will be used at the end of each implementa- 
tion phase to evaluate system performance and to deter- 
mine whether major investments in planned activities 
should be made or not. That is, should we proceed as 
planned to the next phase of implementation. 

The milestones in the plan for 1990 and 1991 are 
listed in table 1 . With reference to milestone 3, the FAA 
Rotorcraft National Survey is complete, and the publica- 
tion of the survey results is expected soon. These data will 
help the FAA improve the services it provides to system 
users, as well as improve rotorcraft forecasts, which serve 
as a foundation for planning and developing future 
strategies. The other milestones include improving the 
public image of rotorcraft, defining heliport networks 
capable of supporting various rotorcraft applications, 
especially scheduled passenger service, and beginning 
; ;parations for tilt-rotor demonstration. I would like to 
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Figure 1 . RPM implementation phases. 
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Table 1. RMP milestones: 1990-1991 


All vertical-lift aircraft Civil tilt rotor 

1990 

1 . Rotorcraft focal point 

2. Rotocraft public image program 

3. Program data systems 

1991 

4. Heliport networks defined 6. CTR demonstration sites chosen 

5. Rotorcraft simulator certification criteria 7. Route structure guidance 


Table 2. RMP milestones: 1992-1993 


All vertical-lift aircraft 

1992 

8. Rotorcraft public image improving 

9. Rotorcraft TERPS complete 

10. Initial helicopter route charts 

1993 

13. Heliport networks operating 

14. U.S. helicopter sales grow 

15. Scheduled helicopter service 


Civil tilt rotor 


1 1 . Funding for vertiport development or improvement 

12. Civil tilt-rotor pilot program 

16. Route network complete (CTR demonstration) 

17. Operator chosen for CTR demonstration 


add here that a recent slip in the military’s V-22 develop- 
ment schedule has necessitated a similar slip in the civil 
tilt-rotor development. Rescheduling some of these mile- 
stones will be necessary as a result. They will be accu- 
rately reflected in the VFPP and in the next revision of the 
RMP. 

Table 2 shows the milestones for 1992 through 1993. 
Activities during this period will include developing suffi- 
cient heliports to establish one or more networks, com- 
pleting preparations for a civil tilt-rotor demonstration, 
and operating schedules for helicopter service. In addi- 
tion, work and emphasis on rotorcraft TERPS will be 
completed; emphasis on improving the public image of 
rotorcraft will continue. This phase of the plan focuses on 
operations, support, and enhancements. It will also deter- 
mine whether activity levels wanrant commitments to 
expand significantly the use of vertical-flight aircraft as a 
NAS capacity enhancement tool. Specific accomplish- 
ments will include adding to and improving heliport/ 
vertiport networks and evaluating the success of heli- 
copter passenger services and the tilt-rotor demonstration. 

The overall objective of this phase is to establish 
100 public-use heliports and vertiports by the year 2000. 
Milestones leading to that checkpoint might include certi- 
fication of the civil tilt-rotor for passenger operations, the 


beginning of scheduled intercity passenger service by 
vertical-lift aircraft, and public-use heliports/vertiports in 
all major hub metropolitan areas. Reaching any of these 
milestones would constitute an impressive achievement 
for vertical flight and mark a significant departure from its 
current applications in NAS. 

In 1988, members of Congress clearly recognized the 
civil potential of technology advances exhibited by the 
XV-1 5 and V-22 and requested development of a plan for 
integration of tilt-rotor technology into the civil air trans- 
portation system. In response, the FAA assumed the lead 
role in launching the National Civil Tilt-Rotor Initiative 
(NCTRI). A five-point program to speed the introduction 
of tilt-rotor technology into the national air transportation 
system was formally started in August 1988, including 
establishment of a national focal point for tilt-rotor activ- 
ity, the tilt-rotor program office, and a memorandum of 
agreement between the FAA and DoD to expedite acquisi- 
tion of test and engineering data from the V-22 program. 

The NCTRI implementation plan was drafted in the 
fall of 1989 to spell out the actions necessary to success- 
fully implement the initiative. Included in that document 
were the tasks and projec's to be carried out, a tentative 
schedule of major milestones, and preliminary cost esti- 
mates. In the NCTRI implementation plan, all of the 
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program tasks were grouped into four elements, or pillars, 
supporting the accomplishment of the demonstration proj- 
ects and full integration of the CTR into the national air 
transportation system. These four pillars were aircraft 
development, public acceptance, infrastructure, and 
certification. 

A series of six major milestones was spelled out in 
the plan, beginning with preparations for a civil opera- 
tional demonstration period and ending with full integra- 
tion into the NAS in December 2010. Critical factors 
affecting the success of the tilt-rotor program included 
congressional support, completion of the V-22 full-scale 
development, test, and evaluation program, and early 
industry and operator commitments. Other important 
information in the plan included a list of roles and respon- 
sibilities by office or organization, costs to government 
and industry, both in terms of yearly expenditures and 
cumulative estimates, and alternative aircraft development 
options that could be used to achieve the tilt-rotor devel- 
opment if the V-22 program was interrupted or 
discontinued. 

Let’s discuss in some detail the VFPP. The purpose 
of the plan is to ensure a hospitable environment when 
industry presents a feasible vertical flight initiative. Also 
it will develop detailed project plans for the period 1991 
through 1994, which is the Phase 1 period; outline 
planned activities for 1 995 through 2000, the Phase 2 
period; and incorporate the contents of the RMP, the 
NCTRI implementation plan, and data from other appro- 
priate plans into one comprehensive document. The pri- 
mary objective of this plan is to make it possible to track 
project status and costs accurately and continually, some- 
thing we are not now able to do. In this way, we will 
always know where the program stands. In addition the 
VFPP will provide cross-plan coordination, eliminate 
overlaps and gaps in existing plans, define schedules and 
resource requirements, and establish roles and responsi- 
bilities for the various participants in the plan. The plan 
will be organized in this format, with the bulk of the 
information contained in the project plans for Phase 1. 

Increasing the role of vertical flight in the national 
transportation system is a cooperative venture requiring a 
successful partnership between government and industry. 

It is the government’s role to create and enhance the cli 


mate in which the rotorcraft industry can continue to 
expand and realize its full potential, but it is up to the pri- 
vate sector to take advantage of opportunities to achieve 
commercially successful rotorcraft services. The plan will 
be prepared by using a matrix-type organization. The ver- 
tical flight special program office will be the overall pro- 
gram coordinator, and the matrix offices will be respon- 
sible for providing project managers, for project plans, 
and for project reporting. Primarily, the types of inputs 
needed from project managers are schedules, resources, 
and project status reports. 

The plan will be updated yearly. In addition, quarterly 
status reports will be required from the managers, and 
quarterly meetings will be held to discuss problems and 
unresolved issues. The management of the plan will con- 
form to the agency guidelines promulgated for program 
management. In this case under the line organization of 
ASD and ARD, the director of the Vertical Flight Pro- 
gram will serve as program manager. That office will 
have overall responsibility for assembling, monitoring, 
and coordinating the plan. Relationships with the various 
matrix team members will be in accordance with written 
operating agreements. 

Vertical flight project manager will supply project 
details to the Vertical Flight Program Office for inclusion 
in the plan. They will be supplied with a sample format 
for submission of their input. 

Finally, the Vertical Flight Program schedule is 
shown below. 

1. Brief Associate Administrators Mar. 11-15, 1991 

2. Brief and train project managers Mar. 18-22 

3. Develop project plan data sheets Apr. 5 

4. Review/modify project sheets Apr. 12 

5. Prepare integrated schedule Apr. 26 

6. Prepare resource annex Apr. 26 

7. Deliver office-level draft May 10 

8. Deliver associate-level draft Jun. 14 

9. Final plan approval Jul. 19 

It is out of date for developing the plan itself. We finished 
the last briefing to the associates on April 19, so that item 
(1) is out of date. We still hope to meet the publication 
date for the first plan, which is the end of July. 
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Peter V. Hwoschinsky is Technical Manager of the FAA’s Vertical Flight Program. 
He was program manager of the FAA’s Aircraft Separation Assurance Program, the 
Aircrew Performance Enhancement and Error Reduction Program, and the Rotorcraft 
Technology Program. He earned bachelor and masters of science degrees and the 
advanced degree of Engineer of Aeronautics and Astronautics at the Massachusetts 
Institute of Technology. He has published eleven training manuals on aeronautical 
decision-making and pilot judgment training. 
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4. SIMULATORS FOR CORPORATE PILOT TRAINING AND 

EVALUATION 


CURT TREICHEL 


First, I would like to thank Bill Larsen, Vickie 
Gardner, and their team for organizing this seminar and 
workshop. I would like to thank each of you for being 
here to share your expertise. And I would like to give spe- 
cial thanks to all of those very talented individuals and 
teams that have given us the simulators we use today. 
We’ve come a long way from the School Link and 
ANT- 18 Blue Box. 

You know, I’m kind of surprised this meeting 
received approval to be held in the San Francisco Bay 
Area, what with all of the faults around here. Apparently 
we accepted the notion that while the experts continue 
trying to improve the earthquake tolerance of the local 
buildings and highways, the area’s many good characteris- 
tics make it a very desirable place to visit, work, and live. 
If only Greg McGowan had so much luck getting 
approval for his simulator — even though they may have a 
few faults. 

I first became familiar with “simulators” for pilot 
training and evaluation when I started instructing at the 
University of Illinois, Institute of Aviation, in 1968. There 
I learned to use a School Link and ANT- 18 Blue Box in 
conjunction with a classroom, chalkboard, and an 
Aeronca CH7FC airplane to train and evaluate candidates 
for the Private Pilot Airplane Certificate. Shortly after 
arriving at Illinois, we acquired several Link General Avi- 
ation Trainers, or GATs, to add to our inventory of learn- 
ing resources. These GATs even had communication 
radios, VORs, ILS, and ADF. Now that was progress! 

Next we replaced the CH7FC Aeroncas with brand new 
modern Piper Cherokee 140s, which also had modern 
radios, including VORs, ILS, and ADF. More progress! 

At Illinois, we also modified the program to require stu- 
dents to train in pairs, so that for every hour of experience 
they gained at the controls, they spent another hour in the 
back seat watching and learning as the other student 
received training. More good progress! 


I left the University of Illinois in 1979 to join the 
United Technologies Corporate Aircraft Department. 
During my 12 years with UTC, I have observed our pilots 
receive simulator training and evaluation for the Beech- 
craft King Air, Cessna Citation, Rockwell Sabreliner, 
Gulfstream III, Gulfstream IV, Boeing 737, Boeing 727, 
and the SK76 helicopter. Talk about progress, I was a part 
of it now! 

United Technologies is a firm believer in the crew 
concept, utilizing cockpit resource-management philoso- 
phies all the time. All of our pilots complete the United 
Airlines/Scientific Methods Cockpit Resource Manage- 
ment course and they also participate in FlightSafety’s 
Practical Cockpit Management programs. The progress 
continues! 

UTC presently operates 10 aircraft, including 
2 SK76Bs, 2 Cessna Citations, 4 Rockwell Sabreliners, 

1 Gulfstream III, and 1 Gulfstream IV. All of our pilots 
are assigned to fly two different types of aircraft, the 
result being that our 16 SK76B pilots also fly the Citation, 
Sabre, or Gulfstream as their other aircraft. Most fly the 
SK76 and a Sabre or Gulfstream to provide each of our 
pilots with one “go somewhere far and fast aircraft” and 
one”go slow and come home every night” aircraft. 

Several years ago when we reduced our fleet size, we 
sold some fixed-wing aircraft, including the B-727 and 
B-737, and increased our SK76B “fleet” from one to two. 
We had two options: lay off eight very experience fixed- 
wing pilots and hire eight helicopters pilots or train those 
eight fixed-wing pilots to also be helicopter pilots. Keep 
in mind these eight airplane pilots all hold the Airline 
Transport Pilot Certificate, Airplane Multi Engine Land, 
with Type Ratings in at least several jet aircraft, and thou- 
sands of hours of experience. Well, we did the right thing. 
We developed a program, in conjunction with Flight- 
Safety, to cross-train those eight pilots onto the SK76B, 
joining the eight pilots already flying both fixed and 
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rotary wing. The fixed-wing-to-SK76-helicopter program 
is shown below. 

We encountered two situations during the program 
that suggested our progress in pilot training and evalua- 
tion had taken three giant steps backward. The first was 
learning that these pilots could not earn their Helicopter 
Instrument Add On Rating in the SK76B simulator. Now 
here’s a simulator with every gadget our aircraft has — just 
what our pilots need to know about if they are going to fly 
IFR in the SK76. But. . .oh no. . .the SK76B simulator is . 
not approved for this. In fact no exemption for this has 
ever been granted for even an airplane simulator. So 
there we were, professional ATP fixed-wing pilots, 
thrashing about in a Hughes 300 helicopter for two more 
weeks (most of that time trying to get somewhere where 
the necessary Navaids could be found) earning a Heli- 
copter Instrument Add On. No EADI, no EHSI, no 
DDAFCS, no EEC, not much of anything relevant to our 
IFR needs. 

And do you know, that Helicopter Instrument Add 
On qualified those guys to fly IFR in any number of other 
types of helicopters, most of them far more complex than 


the Hughes 300. Now, let me tell you — that SK76B simu- 
lator is certainly as useful as a Hughes 300 for training 
and evaluating a pilot earning a Helicopter Instrument 
Add On rating, especially since the rating is category- and 
class-generic, and not specific to just one type of aircraft. 
So, while those of us in this room were busy “studying the 
issue,” those eight pilots and their passengers were short- 
changed. They were not provided reasonable access to 
modern technology. 

The second suggesting of a definite lack of progress 
in recognizing the value of today’s simulator for pilot 
training and evaluation was when we learned they could 
not take their ATP Rotorcraft Helicopter Add On flight 
check in the SK76B simulator. 

Those eight pilots have regularly attended FSI pilot 
recurrent-training twice a year, once for their airplane 
(Citation, Sabre, or Gulfstream) and once for the SK76B. 
Each session includes 3 to 5 days of very thorough class- 
room and simulator training. Operationally, they are fly- 
ing both left and right seat, VFR and IFR, out of such 
places as the several very tight Manhattan heliports and 
the very busy New York Kennedy and LaGuardia airports. 



Task 

Location 

Weeks needed 

1 . 

Instrument written exam 

East Hartford (Rentschler Airport) 

1 

2. 

Commercial add on 

Hughes 300 for about 5 weeks (and classroom) 

Vero Beach, Florida 

5 

3. 

Instrument add on 

Hughes 300 for about 2 weeks (and classroom) 

Vero Beach, Florida 

2 

4. 

SK76 familiarization 

SK76 exterior and interior familiarization 
SK76 familiarization flight 

East Hartford 

1 

5. 

SK76 pilot initial 

SK76 simulator and classroom 

West Palm Beach International 

2 

6. 

SK76 line checks 

SK76 route familiarization including heliports, helipads, 
helistops, ATC, navigation 

East Hartford 

Total 

1 

12 

7. 

Left seat SIC only 

Flying about half of the flights left-seat 


6 months 

8. 

SK76 pilot recurrent 

SK76 simulator and classroom 


1 week 

9. 

10. 

Left or right seat SIC 

Flying about half of the flights right-seat 
ATP rotorcraft/helicopter add on written 


1 8 months 

11 . 

SK76 Pilot recurrent 


1 week 


SK76 simulator and classroom 
1 2. ATP rotorcraft/helicopter SK76 type rating check 


46 


I 


Corporate Use of Simulators 


After 2 to 3 years of this experience, they are more than 
ready to add the Rotorcraft Helicopter Category and Class 
to their Airline Transport Pilot Certificate along with the 
SK76B Type Rating. Ideally, they should be able to com- 
plete their flight check while attending the FlightSafety 
SK76B Pilot Recurrent program, where a professional 
instructor/examiner could observe all of their cockpit 
resource-management and flying skills. In fact, in the 
normal 9 hours of simulator flying that a crew does during 
recurrent training, the vast array of IFR situations and sys- 
tems malfunctions far exceeds what could ever be done in 
the aircraft. 

Once again, while we continue to “study the issue,” 
those eight pilots, and many more like them, along with 
their passengers, are being shortchanged because they 
cannot complete their ATP/Type Rating checks in the 
simulator. 

Keep in mind that conducting the check in the aircraft 
(1) requires putting a $5 million dollar aircraft out of pas- 
senger service for half a day; (2) flying as much as 
4 hours to get to the examiner, take the ride, and fly home 
(cost: $5000+); (3) causes the pilot to be unable to do 
many of the very important tasks normally done in the 
simulator; and (4) places both the aircraft and its occu- 
pants in a high-risk situation. 

Now wait until you hear what the SK76B is not 
approved to do for the ATP/Type Rating flight check. 
Certification credit is not approved for the following: 

(1) 360 turns at a hover, (2) normal takeoff from a hover, 
(3) manually flown precision approach, and steep 
approach to, and landing at, a helipad. Remember the 
7 weeks of a Hughes 300 flying that occurred 2 to 3 years 
earlier? Well, they hovered and they hovered and did 


pedal turns then. I cannot imagine any pilot with the expe- 
rience necessary to apply for the helicopter ATP not being 
able to hover, do pedal turns, fly a steep approach, or do a 
simple ILS approach. 

Folks, we must focus on the many values the simula- 
tor has to offer, and stop dwelling on its few shortcom- 
ings, especially when those shortcomings are not relevant 
to the particular level of training and evaluation at issue. 

The SK76B simulator has many, many advantages 
over the aircraft, for training, or, conversely, the aircraft 
has many shortcomings when compared with the simula- 
tor. Yet, we are very willing to approve training and eval- 
uation in the aircraft while at the same time being 
extremely critical of the simulator. 

We must also not forget that any training resource, be 
it chalkboard, textbook, aircraft, or simulator, is only part 
of a total training and evaluation program, and the 
instructor/examiner is generally the critical difference 
between a good program and a poor program. If only the 
instructor/examiner received as much attention and fund- 
ing as the aircraft and simulator do. 

Let me summarize with the following four points: 

1 . We should continue to design and build highways 
and buildings that are earthquake-proof. 

2. We should continue our quest for the perfect 
simulator. 

3. We should accept the present-day San Francisco 
Bay Area, even with its faults, as a very desirable place to 
visit, work, and live. 

4. We should accept the present-day simulators, even 
with their faults, as at least equal to, and in many cases, 
superior to the actual aircraft as a pilot-training and evalu- 
ation resource. 



Curt Treichel is manager of training at the Corporate Aircraft Department of United 
Technologies, Inc. He is responsible for the training of 130 administrators, aircrew, 
cabin attendants, maintenance technicians, and managers in a corporate flight 
department operating 12 aircraft, including the SK76B. Mr. Treichel has studied 
transfer to training from simulators to aircraft for the University of Illinois Institute 
of Aviation. He has a B.S. degree in business administration from Defiance College 
and an M.S. degree in vocational and technical education from the University of 
Illinois, He has Airline Transport Pilot certificates for Airplane Multi-Engine Land 
and Rotorcraft-Helicopter, and ATP-type ratings for the CE-550, NA-265, and SK76. 
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5. TRAINING EFFECTIVENESS ASSESSMENT: WHERE ARE WE? 

GREG MCGOWAN 


I would like to thank NASA and the FAA for allow- 
ing FlightSafety to participate in this workshop. What I 
hope to do is set a framework for your participation in the 
panel discussion that we will be doing on Thursday. I 
know with all the presentations going on there are a lot of 
questions you will not have the opportunity to ask or get 
answers to. I think the panel discussions will provide an 
opportunity for that kind of participation. 

Concerning the Workshop itself, I look at it from an 
objective standpoint. Even though we are focusing on 
simulators and on certification criteria and so on, I think 
we should be looking at how to provide tools for instruc- 
tors and companies like FlightSafety, to better serve end 
users like Curt Treichel and Jerry Golden, for example, in 
providing safer pilots and safer aircraft operations. 

As an overview to this we will take a look at an 
introduction and historical review, not spending much 
time on the first three or four points. From a historical 
perspective, I think it is important to see where we have 
come from and why we got started in the first place and 
where we are now. Because we are using commercial 
helicopter simulators, we have to ask, how efficient are 
they and how can we optimize their utilization? 

As far as where we are, I think we have to define that 
question in terms of a reference point. We have been beat- 
ing around the bush about this a little bit, but I think this 
Workshop is really concerned with — or at least I am con- 
cerned with — commercial helicopter simulators in the 
United States. I had an opportunity to fly the LHX check 
simulator about 2 months ago. That simulator is a com- 
pletely different animal. It represents some great technol- 
ogy, and interesting things are going to come out of it. 
However, I think the emphasis here must be on commer- 
cial helicopter simulators. We also need to define the 
environment. Are we talking about cost, safety, fidelity, 
and effectiveness of training? I think those are important 
issues that need to be looked at. No one of those issues is 
more important than another; it depends on the end users’ 
requirements, on what is most important to them. I would 


like to take a look at some of those things today briefly, 
and in more detail in the panel discussions. 

From a historical review standpoint, why did we even 
get involved with commercial helicopter simulation? Back 
in the 1970s, Bell Helicopter and Sikorsky Aircraft 
decided to build, for the first time, a commercial heli- 
copter that was not merely a military derivative, the 
Bell 222 and the S-76, respectively. The customers they 
perceived to make up the market for those helicopters 
really consisted of two groups, corporate and offshore, or 
corporate and utility. Certainly there were segments of 
both of those markets that were going to require a simula- 
tor in training the pilots and maintenance technicians for 
those aircraft. And it was the position of both Bell and 
Sikorsky that it would be necessary to have a simulator- 
based training program as part of the overall marketing 
effort for those helicopters. 

That is why the first commercial U.S. helicopter sim- 
ulators were built. You might say the helicopter manufac- 
turers, therefore, are the ones who provided that initial 
impetus to simulator development. But it is really the end 
users, the Curt Treichels and Jerry Goldens, the people 
who use the simulators who drive that market. Without 
that market requirement, the manufacturers would not 
have spent the money on developing simulators. 

Initially, when a simulator-based training program is 
part of a manufacturing agreement, such as we have with 
Bell and Sikorsky, the first course to be developed is ini- 
tial training.which is then quickly followed by recurrent 
training. 

I am proud to say that we are now getting into what I 
call generic training, using simulators that are designed 
for specific aircraft, but using them in a generic way. For 
example, there are the Emergency Medical Service (EMS) 
helicopter pilot recurrent course and the instrument 
refresher courses. We have pilots flying Augustas and 
small Bell products, as well as the Aerospatiale products, 
which don’t have simulators, enrolled in courses in which 
they are using an S-76 or Bell 222 simulator to get as 
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much as they can out of a simulator-based training pro- 
gram. They are practicing things like crew coordination, 
cockpit management, and instrument procedures. The 
technology is developed to the point that we can duplicate 
the actual aircraft, but we tend to forget the other applica- 
tions that we used years ago in the Links and Dehenel 
trainers and the training devices, which are still applicable 
in the current generation of simulators. 

We are really only talking about three simulators. We 
have two aircraft for which there are certified simulators, 
those being the 222 and the S-76B. There is also a third 
training device out there that did some ground breaking 
on its own from an exemption standpoint, and that is the 
S-76A. 

More accurately, the S-76A is for all practical pur- 
poses a training device. It is the most sophisticated train- 
ing device I have ever seen. 

At the end of 1990, there were 174 Bell 222s, and 
319 S-76 aircraft worldwide. A total of 3,747 pilots were 
trained in the Bell 222 and 5,096 were trained in the S-76; 
that is, in all types of training between 1980 and 1990. 

The check ride numbers are 354 for the Bell and 2,333 for 
the S-76. The reason I point this out is because there are 
significant opportunities for data collection here. There- 
fore, these two pilot training devices were used to train 
almost 9,000 pilots and to give about 2,700 FAR checks. 

A breakdown of those check rides shows virtually all of 
the 61.57 instrument competency checks (1,296) were 
done in the Bell 222 simulator. There are reasons for that I 
don’t need to go into, but the primary one is that the 
61.58 PIC check is not required in the Bell 222; as a 
result, the best thing you can do is a biennial flight review 
or instrument competency check. 

The 61.57 instrument competency check totals 
( 1 ,296) are from a combination of the Bell 222 and the 
S-76. The low numbers of 135.293 (129) and 135.297 
(121) checks are a result of our doing them for only a 
couple of years. 

Regarding the commercial helicopter simulators — 
without going into a lot of detail, I certainly will provide 
syllabuses for any of the courses to anyone who wants 
them; just give me a call and we will mail them out. 

The initial training course is 2 weeks long. It was cer- 
tainly the first course developed for either of the S-76 sim- 
ulators, or for the Bell 222, for that matter. Most of the 
recurrent training courses are 4 days long. We do have 
specialized courses of 3 and 5 days for certain operators 
and special requirements. One of the points I want to 
make here, though, is that before we had our first exemp- 


tion, our generic courses, things like the recurrent training 
and the initial training we were doing, were well attended, 
even though the pilots were getting absolutely no credit 
whatsoever. I think that that is an important point for all 
of us to remember: the end user, the pilot, the operator, 
the company, recognized the value of the training, and 
they were willing to pay for it in many cases without any 
checking credit, without any training credit whatsoever. 

On the other hand, I think we also need to realize that just 
because they have been doing it does not mean they are 
going to continue to do it, especially as costs go up. 

Figure 1 shows what we call a pilot proficiency 
record. Actually, it is a five-page document. This is what 
our instructors use to evaluate pilots undergoing training 
and checking at the Center. It is a part of the pilot’s train- 
ing record. The shaded items are those that would be 
required for an ATP check or for a pilot command 
61.58 proficiency check. I believe the regulation reads 
that the same items and maneuvers that would be done for 
the initial issuance of type-rating would be required or 
recommended for 61.58 pilot proficiency check. 

The unshaded items are those things pilots are 
required to complete during our course of instruction, 
which, by the way, is FAA approved. They also receive 
what is called a flight-safety proficiency card. It has been 
mentioned that we did so much more than required. For 
example, on engine malfunction, the high-side governor 
failure was mentioned. We have them do high-side and 
low-side governor failures. They cannot do those in the 
aircraft, and it is something pilots make mistakes on. They 
can get that experience only on the simulator. That is what 
simulator-based training is all about. We can talk about 
this more in the panel discussion, if we get a chance. 

A little history of the exemptions might be in order. 
Exemption 4609 was issued in January 1986 (table 1). I 
think we started the request in early 1984, 1 think we first 
had a meeting up in Washington, D.C. It took time, 
because we were breaking new ground; but we eventually 
got it for the S-76 training device and for the Bell 222 
simulator, with which we do the PIC check and flight 
review. Numerous prerequisites and recency-of- 
experience requirements are stated. 

In almost all cases, even with fixed-wing simulators 
in which checking or training are done, an approved 
course of instruction is included. You don’t just go out 
and use these simulators to do a check ride. There is an 
approved program of instruction; the same is true for this 
exemption. For example, aeronautical experience from 
( .61 requires 50 hours in the last 12 months, 5 hours 
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Figure 1. FlightSafety pilot proficiency record. 
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Table 1 . Exemption 4609 


Table 2. Exemption 5067 


1. Exemption issued 28 January 1986 

2. Applicable to S-76 training device; Bell 222 simulator 

3. Prerequisites/recency requirements: 

Approved training course 
Aeronautical experience (61.161) 

50 hours preceding 12 months in type 
5 hours PIC last 60 days, make and model 
3 takeoff and landings last 90 days 

4. Amended 23 June 1988 to include S-76B level C 
simulator 


PIC, and three takeoff and landings in the last 90 days. 

The customer base we are addressing has no problem 
meeting these. It was amended in June to include the 
S-76 simulator. I am using those terms loosely because 
they don’t really apply. We cannot call it a level C; it is an 
approved helicopter simulator. That is the proper termi- 
nology, but if you use it people ask you so many questions 
it is better to call it a level C and not have to explain all 
this. 

Exemption 5067 was issued 29 June 1989; it is 
applicable to level C simulators. It is an outgrowth of the 
approval we got with the simulator, and it is approved for 
conducting the checks shown in table 2. Those pilots 
undergoing these checks have to certify that they have, for 
example, done three slope takeoffs and landing within the 
last 90 days. This is not a real big problem when you 
consider that runways are usually crowned and therefore 
have some degree of slope. The other prerequisites 
include 100 hours in the preceding 12 months, 10 hours in 
the S-76, 50 hours in the preceding 6 months, visual 
inspection, 360° pedal turn in hover, normal takeoff from 
hover, manual flown precision approach, and steep 
approach and landing. 

As soon as an exemption or regulation requires that a 
pilot do anything in an aircraft, with respect to checking 
or training, you will eliminate a certain segment of that 
population that would otherwise train in the simulator. 
They won’t train in the simulator because it costs you 
about $2,500 an hour to tly the aircraft. And it can cost 
even more if travel is involved in getting to the examiner. 
So a lot of these decisions are based very much on eco- 
nomics. That’s something that we need to talk about in the 
panel discussion. 

A question that really needs to be asked is how effec- 
tive are commercial simulators? Objectively, I think more 
research is needed. That is one reason I showed you the 


1. Exemption issued 29 June 1989 

2. Applicable to S-76B level C simulator 

3. Approved to conduct the following: 

61.56: 24-month flight review 

61.57: Day/night landing currency 

61.58: 1 2/24-month PIC check 

61.163: ATP rotorcraft (90%) 

135.293: Recurrent testing 

1 35.297: Instrument proficiency 

4. Prerequisites/requirements: 

Approved training course 

Three slope takeoffs/landings 90 days 
100 hours preceding 12 months (10 hr S-76) 
50 hours preceding 6 months (5 hr S-76) 
61.163 ATP/add-on, flight test in S-76 
Visual inspection 
360° pedal turn in hover 
Normal takeoff from hover 


Manually flown precision approach 


proach and landing at heliport 


numbers that we have. The people are coming to train, 
and as a result the opportunities for collecting data are 
there. At FlightSafety we certainly are not experts at col- 
lecting data. I don’t know what kinds of questions to ask 
these people or what kinds of maneuvers to ask them to 
see and duplicate. 

There is one thing I want to mention when talking 
about duplication. When we are evaluating these simula- 
tors and we go out and fly the aircraft and we come in and 
fly the simulator, we need to fly that helicopter at night. 
We need to be dping those 360° pedal turns in a hover at 
night over a runway similar to what we have in the air- 
craft, or in the simulator. I realize in some cases we are 
looking at breakout forces and things that don’t really 
make a difference visually. But when you are subjectively 
evaluating the overall quality of a simulator I think it is 
unfair to go out in the daytime with all the daytime visual 
cues and compare it with a night visual system. 

Subjectively though, I think the simulators are very 
good for a number of reasons. We have the data, we have 
the pilots, and we have a lot of FAA pilots that have gone 
through training who can tell you about the level of 
instruction, the kinds of things that can be simulated, the 
maneuvers that they can do in the simulator and then 
compare with the actual aircraft. We have some people 
say the simulator doesn’t hover right, and we have others 
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who say it hovers just like the aircraft. That is why we 
need to collect more data and find out what the weak- 
nesses and strengths are. 

We also need to keep in perspective the overall idea 
that there is a lot more positive to be said about the simu- 
lator than negative. The article I mentioned earlier about 
the helicopter that went down in the river off of the Wall 
Street heliport is a good example. This is a quote from the 
pilot, Sandy Kaplan. “The engine quit on departure. We 
didn’t have enough power to continue. We just went 
down, just like we practiced at FlightSafety — you bet!” 
That is an example of the benefits they gained from train- 
ing received in a simulator that they could not have 
received in the aircraft. 

Lastly, how are we going to optimize the effective 
utilization of helicopter simulators? We already talked 
about some of them. I think we need to look at the regula- 
tions and to have an opportunity for giving the two differ- 
ent types of check rides so you can substitute things that 
can be done in the simulator for things that perhaps cannot 
be done in the aircraft. In other words, maybe one low- 
side governor failure and one high-side governor failure 
and an engine fire could equal one 360° pedal turn in a 
hover — for lack of a better example. We need to look at 
the philosophy of simulator use. 

That includes looking at things such as I just men- 
tioned. We need to do a better job of training our instruc- 
tors. We have problems as a company, as a simulator 
trainer company that uses instructors for simulator train- 
ing. We need to better educate those instructors, we need 
to do a better job of training them in cockpit resources 
management, in how to do a better job of debriefing to get 
as much as we can out of the training tools. I refer also to 
cost. For example, Jerry Golden and Curt Treichel — they 
are the one who ultimately decide whether they will use 
the $ 10-million and $ 12-million simulators that we train 
with. 

MR. McDANIEL: By the way, I flew that approach 
to Wall Street and landed in the water as well. I did it in 
his simulator a couple of weeks ago. We practice doing 
those things and we did it successfully the first time we 
tried it in the simulator. And after going through the pro- 
cedures with instruction, we did high-side governor fail- 
ure, we did low-side, tail-rotor failures, fixed pitch, all of 
those things. Quite frankly we were not always successful 
on our first attempts on those things in the simulator. But 
anyhow, the thing is, there is some excellent instruction 
out there that is available with this kind of thing. As we 
said, we had a number of discussions but active conversa- 


tions on the usefulness of it, and I am convinced that it is 
a very useful training instrument and something that we 
need to get credit for and bring into the system. That is 
really why we are here. 

MR. CARVER: Just three observations on that very 
excellent rundown. There is a lot of thought in what you 
said. 

First of all, as far as training and checking are con- 
cerned, everybody wants credits for training devices or 
simulators or whatever. Of course the observation of pilot 
regulators is that pilots need more training than that which 
a regulator requires, so as long as training is not negative, 
then most regulators would support what you have just 
suggested, that is, without necessarily having credits, 
because it is the commercial public transport company 
that is responsible for the pilot training, and what the reg- 
ulator wants is really a snapshot of something at the end. 

As far as effectiveness is concerned, there are one or 
two other points. Effectiveness depends on the fidelity of 
the simulator, on its maintenance records above all. There 
is a thought there with regard to the complexity of the 
device and what effort the company is willing to put into 
its maintenance, and the ability and imagination of the 
instructor-examiners. I definitely agree there with you. 

And finally, I am not a rotary pilot, but as far as the 
simulator is concerned, rotary really requires more pilot- 
ing skills, so I think we have to be careful when giving 
licensing credits to a simulator. But certainly the generic, 
the human factor is certainly an area in which it is useful. 

MR. McGOWAN: Those are good points. I hope you 
come to the panel discussion because those are the kinds 
of things I think we need to talk about. That is the whole 
ideas of this presentation: to whet your appetite for that 
panel discussion. 

MR. LOMBARDO: When I first went to work for 
FlightSafety back in 1979 and 1980 in the King Air pro- 
gram, one of the things that I was very dismayed to dis- 
cover was that the training for the instructor was very 
minimal, and there was an assumption, which it appears 
will continue through the 1990s, that if you are a good 
pilot you must be a good instructor. And what Curt will 
testify to is we did the job with the Blue Box and we can 
do a better job with more sophisticated equipment, but 
what industry needs, and I have had a devil of a time try- 
ing to convince anybody of, is guidelines for a structured 
training program for people who are going to instruct in 
simulators. 

Typically what happens is we find somebody who is 
typed in the aircraft or has experience in the aircraft and 
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we put him in the box and assume he knows how to teach 
in a simulator. These people tend to fall into one of two 
categories: (1) those who use the simulator exactly as they 
use the aircraft, in which case they underutilize the 
equipment; or (2) those whose approach is let's see what I 
can do to them today, who overload the students. I am not 
a helicopter pilot; I am a fixed-wing pilot. Still, I would 
say that what needs to be done in the helicopter industry is 
to develop the guidelines for, or formulate a committee to 
put together, a program to teach people how to teach in 
simulators. You can do more with a good instructor and 
less accurate piece of hardware than you can do with a 
highly accurate piece of hardware and a poor instructor. 

MR. McGOWAN: I agree with that last point that 
you made, totally. I will say that more than 4 years ago 
FlightSafety finally recognized part of what you said and 
developed an instructor development course that all of our 
instructors now go through. It is a 5-day course, standard- 
ized, taught in one location in Texas, and all of our 
instructors have to go through it. 

There is a recurrent instructor course. It is not a do-all 
and end-all for the problem you are talking about. The 
Center is also ultimately responsible, through standard- 
ization, to ensure that the instructor is using these tools 
effectively. The FAA also has a part in that. Once you 
become a pilot-proficiency examiner you have to undergo 
check rides and they actually sit in on the check ride or a 
portion of it. A lot of the checks we do are progressive 
checks, and they have an opportunity to criticize or make 
comments on how you are doing your job, whether you 
are doing it effectively or not. These are important things. 
We could have a whole workshop dedicated to the subject 
of instructor training. 

MR. McDANIEL: I agree with your point that a 
good pilot does not necessarily make a good instructor. I 


have known many very good pilots who are not very good 
at instructing. I would say that a good instructor pilot 
probably does have the skills to be a good simulator 
instructor. But there are differences between instructing in 
the actual aircraft and in the simulator and some strengths 
of the simulator, some capabilities of the simulator, make 
instructing in the simulator different from instructing in 
the real aircraft. I think we all appreciate and recognize 
this. I agree, you do need some kind of instructional pro- 
gram for the simulator instructor so he can best take 
advantage of the strengths of simulator use. 

MR. CLENNEY: I agree 100%, because I also have 
been an instrument flight examiner in both airplanes and 
helicopters. When you start giving an instrument flight 
examination in a simulator, you are also now air-traffic 
control, and you have to plan your air-traffic control so it 
will be realistic for your pilot, the pilot is busy, but the 
instructor is busier. So I highly endorse this idea. 

MR. McGOWAN: You are absolutely right. That is 
one of the things that have to be done in your instructor 
training. Probably the most difficult thing to teach an 
instructor is how to think further ahead than he or she has 
ever thought before because you have to be the ATC func- 
tion, you have to be the Center, and you also have to, in 
some cases (for example in the EMS recurrent course) 
play the role of doctor, nurse, or EMT in the back of the 
helicopter during a loft scenario. It is a really busy job and 
it is actually, from a planning standpoint, much easier to 
do in the aircraft, because then you are really at the mercy 
of the system. You either get the ILS approach or you 
don't. In a simulator you have to plan for it. If you haven’t 
done the proper planning, in the simulator there is no sys- 
tem to take care of you. 
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. CURRENT TRAINING: WHERE ARE WE? 

GERALD GOLDEN 


I appreciate very much being asked to speak at this 
simulator workshop. I am here purely as a 135 operator 
and a trainer of pilots. I am not going to even begin to try 
to address the technicalities involved in building, design- 
ing, or certifying a simulator. It’s not my bag of tricks. 
However, I do believe it is very important that operators 
participate in this kind of seminar, because we are going 
to be the ultimate user of the product of this process. And 
by that I mean the advisory circular as well as the simula- 
tor itself. I am probably not going to use all the time allot- 
ted because I have only about three points that I would 
like to make and I can make them fairly short and sweet. 
Then we can go on to something else. 

Initially, what I am going to say may sound like an 
advertisement for Petroleum Helicopters, Inc. (PHI ). But 
it is not intended to be that; it is just an effort to try to 
show you the scope of what we actually do. If you will 
just bear with me you will understand my approach in just 
a second. 

Most of the people in the industry have heard of 
Petroleum Helicopters, but very few understand what we 
do and how we go about getting it done. We have about 
2,400 employees, of whom about 800 are mechanics, and 
about 750 pilots. And we have 17 bases scattered across 
the Gulf of Mexico, from Rockport, Texas, to Mobile, 
Alabama. We operate about 300 helicopters, and we fly 
VFR and IFR up to about 175 miles offshore. The day is 
coming when our nearest IFR alternate will be the 
Yucatan Peninsula. There are oil leases, drilling leases 
that far offshore that have been sold; they are just waiting 
to be drilled. That day is coming. So the world we operate 
in is undergoing constant change, too. We also operate 
10 F-76s in support of EMS base hospital programs. Col- 
lectively, we and our competitors operate approximately 
600 helicopters every day in the Gulf of Mexico, primar- 
ily in support of the offshore petroleum industry. 

To crew our 300 helicopters, which comprise seven 
different makes and models, our 750 pilots require about 
1 ,700 to 1 ,800 check rides per year. Those are recurrent 


training check rides, and have nothing to do with transi- 
tion, upgrades, or initial — that sort of thing. Just the recur- 
rent training of the 750 pilots. Two hundred fifty pilots 
operate under instrument flying rules. These 250 IFR 
crewmen receive about 500 check rides, each of which 
takes about an hour and a half. Some are quite a bit 
longer, depending on where the aircraft is based and 
where the precision approach is located. This equates to 
about 750 flight hours annually just to maintain our IFR 
crews. 

To give you an idea of the cost to us as a user, the 
average direct operating cost of the aircraft is about 
$1,750 per hour. That does not include the costs of our 
facilities, insurance, or other expenses involved. The 
recurrent training needs just described cost about 
$1.3 million per year. This figure does not address the 
FAR 61 recent-experience requirement; this is purely the 
Part 135.297 check ride. And we are required in many 
cases to maintain this Part 61 recent-experience. I am 
talking about the 6, 6, and 6 (6 approaches, 6 hr 
instruments, in last 6 months). 

Where are we now with our training needs? Virtually 
as we speak, we are in the process of upgrading 10 crew- 
men to the status of IFR, SICs. To do that will take about 
120 flight hours, an average of 12 hours each. That is 
about $21,000 apiece, or a total of $210,000 in direct 
operating costs alone. This summer we are going to 
upgrade an additional 18 pilots, 12 of whom will go to 
PIC standards and 6 to SIC standards. This is going to 
require approximately 216 hours, at a cost to us of about 
$378,000. 

The point I am trying to make is that we do this with- 
out a simulator. I wish we were using a good, authorized 
simulator. Obviously it would provide not only what I 
think would be a better trained crew, but it would go a 
long way toward reducing our costs. 

Just as a note of interest, I am working with Flight- 
Safety right now to try to purchase about a 100-hour block 
of simulator time at our Cleveland base. We won’t get 
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simulator training credit from the FAA, because I am 
going to use a BE-200 simulator, which is all they have. 
The truth of the matter is it will cut down my time, it will 
cut down the cost, and I am going to do it whether the 
FAA recognizes it or not. 

This should provide a glimpse of the tip of the ice- 
berg of the training needs that we have at PHI. If you stop 
and think about that, with 600 helicopters out there, of 
which we have 300, obviously this is only about half the 
cost that is involved. So in answer to an earlier question 
about the potential use of a simulator like this, we would 
probably use it about 1 ,250 hours a year if available and 
affordable. By available I mean fairly accessible, at a 
nearby location. 

Before PHI leases or operates any simulator on a reg- 
ular basis, there are criteria that the simulator has to meet. 
This is because our costs are already so high that the use 
of a simulator must help me reduce those costs, as well as 
provide that extra level of training. I understand fully that 
a pilot who has been trained and retrained in a simulator 
gets many opportunities to do a lot of extra practice of 
various maneuvers. I have been through the S-76 over at 
FlightSafety; it is a phenomenal piece of machinery as far 
as I am concerned. 

Any simulator we use must be approved for credit 
toward the training that we do. We do FAR 1 35.293 check 
rides, and it should be possible to do some of that in a 
simulator. But we should be able to do all of the FAR 
135.297 check rides in that simulator. The simulator must 
be practical. By that I mean that in addition to the usual 
IFR features that we think about — the ILS, the VOR, the 
SDFs, the other type approaches — the simulator must 
address offshore flying techniques. 

Specifically I am talking about airborne radar 
approaches, HEDA let-downs and what is referred to as 
offshore standard approach procedure (OSAP). All of 
these approaches use a combination of interface with 
weather-avoidance radar and the LORAN. These 
approaches are fairly commonplace and they are fairly 
simple, but they must he checked in an ongoing check- 
ride program. These approaches are part of the reason 
why our check rides are so long just for an FAR 135.297 
check. If the aircraft is based in Lafayette, Louisiana, it is 
about 40 miles to the Gulf, but it's about 50 miles to a 
place where I can execute a radar airborne approach. I 
have to get over the water to do that. 

I might comment here on the practicality of some- 
thing that was mentioned earlier, the necessity to do 
visual-reference maneuvers. I am not totally convinced 


that the simulator should be able to do a slope landing. I 
do not think it should be able to do a confined area, and I 
am not interested in doing an autorotation. I would not 
dream of trying to do Part 133 external loads in a 
simulator. Those are ground-reference maneuvers, and 
they are maneuvers that are best practiced, in my opinion, 
in a helicopter. Those are specialized procedures. I want 
no part of trying to certify a pilot to do slope landings in a 
simulator. To me, it's just not necessary. I want to do the 
other things, like we said, the high-side and the low-side 
governor failures, things I can't simulate in a helicopter. 

The final criteria that a simulator must meet before 
PHI or any other operator is going to use it, have to do 
with cost. 1 heard mention earlier about $12 million simu- 
lators. I would like to ow n a $ 1 2 million simulator. In 
Lafayette, Louisiana, the use rate would be fairly high, 
because of the number of pilots there. And yet the bottom 
line is this: that 2,000 hours a year is not going to cause 
Greg McGow'an to put a $ 1 2 million simulator in 
Lafayette. These simulators are simply priced totally out 
of the reach of operators such as ourselves. 

I cannot afford to buy. even over the long term, a 
$12 million simulator. 1 would like to have one nearby 
that I could use, though. Contrary to what our monthly 
lease rates might indicate, there is not a whole lot of 
markup in offshore helicopter transportation prices. 

The final point that I want to make is concerned with 
the advisory circular itself. The stated purpose of the advi- 
sory circular is not to mandate, but to provide a way to do 
things. Well, there have been advisory circulars over the 
years that were designed to be just that, advisory, that 
wind up being regulatory because there is no other 
accepted way to do w'hat those advisory circulars approve. 
I am icferring to Advisory Circular 90.80 as a good 
example. 

For a long time we did airborne radar approaches off- 
shore, routinely, day in, day out. Advisory Circular 90.80 
gave an acceptable way to do airborne-radar approaches. 
The truth of the matter was the advisory circular was 
based on a piece of equipment that was not available to 
the public. There was no way we could comply with the 
Circular. Since it was not mandatory we just went about 
our business. But one day the FAA said if you are not 
doing it according to 90.80. you cannot do it anymore, so 
get into compliance. That is the point I am trying to make. 
When the advisory circular is written, you need to put 
yourselves in the users’ shoes so you understand their 
needs, as well what Dick Birnbach said this morning 
about writing the “paper.” He made the comment, “I don’t 
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use them, I don’t fly, I only write the paper.” Well, we 
had better consider the people who have to use the advi- 
sory circular, as well as the simulator itself. That basically 
concludes what I have to say. I will be happy to take 
whatever questions you have. 

MR. McDANIEL: In talking with different people 
about what is required of a simulator and what is not, 
there are questions such as is motion good, is visual good, 
and do you need motion. And in one of your comments 
about the advisability of having a simulator to do slope- 
type operations, autorotations, etc., you mentioned exter- 
nal loads. I guess my question is if the simulator has the 
fidelity, and can do those things, is it of value to have that 
capability? I would take it as a given that, yes, you would 
want to confirm that training capability or the capability 
of the pilot to conduct those operations in the aircraft. But 
is it of training value to be able to do that when it is 
cheaper to do it in a simulator? You made the statement 
that it is of no value to you and that you do not care to see 
that in a simulator. 

MR. GOLDEN: Here we get into engineering. The 
engineering and design features that have to be built in 
that will realistically represent a sling-load are going to be 
phenomenal. The same thing is true of a slope. I don’t 
think what you see in a simulator when doing a slope 
operation is going to have any real bearing on what it is 
like in the real world of helicopters. So how much does 
this cost? It comes down to money. Sure, given enough 
time, given enough engineering, we would come up with 
a simulator that does a fair or reasonable job of simulating 
slope. But is it really necessary for what I need to do? The 
S-76A simulator, which has been available since 1971, 
does a phenomenal job of everything I need to do except 
for the LORAN radar interface. 

MR. HWOSCHINSKY: You said you are a trainer of 
pilots, but you are a trainer of instructors as well. How 
would you envision the use of simulators in training your 
instructors, particularly given the fact they need to know 
the limitations of the equipment itself? 

MR GOLDEN: A good question. I cannot possibly do 
all the training that PHI requires. I have 17 instructors 
working for me and I can tell you that the training of those 
17 instructors is ongoing continually. It is necessary. The 
training they have to go through is nonstop. Training an 
instructor for a simulator is something I have never done 
in a full-blown motion simulator, but I bet Greg 
[McGowan] can tell you about that. 

We did build in-house several years ago what we call 
a 206 procedures trainer; there is only one like it that I am 


aware of. This device is capable of doing hot starts, pre- 
mature light-offs, and fires; you can simulate malfunc- 
tions through the use of switches. It does not fly, and none 
of the flight instruments move. But all the problems asso- 
ciated with starts, in-flight routine, emergency sort of 
stuff, you can do on this simple little device. We had 
problems with the way we were doing things in that simu- 
lator, however. We had to work a pretty good scenario 
just teaching the guys how to use that simple training 
device. 1 hesitate to think how many flight instructors or 
simulator instructors FlightSafety has and how much their 
training bill is just to qualify to maintain those instructors. 
It has to be staggering. 

MR. WARTH: I run into your helicopters all the 
time. 

MR GOLDEN: Not literally. 

MR. WARTH: I was interested in the aircraft operat- 
ing cost you mentioned — the $1700 per hour. Does that 
include instructor time? 

MR. GOLDEN: No, just the operating cost of the 
machine. That was an average for three helicopters. The 
cost for the S-76 is considerably higher than that. 

MR. WARTH: Oh, really? I was interested because 
for our two Coast Guard helicopters we have a cost of 
about $1,200 an hour. Sounds like a similar basis of cost. 
And it is only about $120 an hour for the simulator, so 
there is a big cost benefit for us to use the simulators we 
have. 

MR. GOLDEN: Greg, can I have some of that 
$120-an-hour simulator time? 

MR. WARTH: If you want to fly an H-3 or H-65, 
sure. 

MR. GOLDEN: I don’t. Sorry. 

MR. WARTH: I am also curious about autorotation. 

MR. GOLDEN: In the simulator? 

MR. WARTH: Right. That is a big thing for us. 

MR. GOLDEN: Well, the second time I went over to 
FlightSafety to fly the S-76A, we spent probably 35 or 
40 minutes doing autorotations. I forget the instructor’s 
name now, but he said look, what you do is you descend 
down to 100 feet (this may be correct, it may not be), 20° 
nose up, go down 20 feet, and pull a pitch and land, so I 
did exactly that. Just glued myself to the gauges, went 
down and did what he said and the autorotation was suc- 
cessful. The other person who was with me was a 
full-blown captain who probably had about 15,000 hours 
of flight time. He could autorotate virtually everything 
that flew, but he spent the next 35 or 40 minutes trying to 
get one to come out right on the S-76. 1 maintain that 
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exercise did not help him in that particular respect. I am 
positive about how the man autorotates. He simply was 
not good at flying a simulator. 

MR. WARTH: In that case it was a question of the 
fidelity of the simulator. 

MR. GOLDEN: Probably. I am not all that impressed 
with ground-reference maneuvers in a simulator. But I 
don’t think in any ATP check ride that a pilot who has 
that level of experience should be required to demonstrate 
a 360° turn. 

MR. WARTH: How about entry-level pilots who 
have to do IFR autos? If you are going to train pilots in 
simulators, presumably you are going to train . . . 

MR. GOLDEN: I am talking about the touchdown 
portion of the autorotation. I think that would be best per- 
formed in the aircraft. 

MR. WARTH: I understand. Thank you. 

MR. BOOTHE: I just wanted to mention that your 
experience in the S-76A is probably one of the reasons 
why it is not a qualified simulator. It is a training device. 
That gets to a point that made this morning: the device has 
to support the training or checking maneuver that is to be 
done. We had this same experience with airplanes. I 
remember in one of the first Level C simulators, we could 
not land Phase 2. The instructor said, well, just stay about 
200 feet, pull throttles to auto, pull attitude 8°. We said we 
do not land the airplane like that. I think that is the same 
thing you are experiencing. 

MR. GOLDEN: I would like to respond to that just in 
part. There are so many other things that the simulator 
will do that I think we should be able to receive recogni- 
tion and credit for doing those. Granted, if I have to do a 
touchdown autorotation, which I don’t, by the way, then I 
should be able to do that in the aircraft. But I am not 
required to do that in the aircraft, and therefore I should 
not be required to do it in the simulator for certification 
purposes. 

MR. BOOTHE: I agree with that, I just wanted to 
touch on the four maneuvers. As somebody pointed out, 
maybe they are the wrong four. As regulators we are con- 
servative. Kind of like turtles, we stick our noses out just 
a little bit before we stick our necks out. I think that is 
really what we are talking about with the S-76B. We 
never before in the civil segment qualified a simulator of 


that sophistication. And if you recall, when we started 
with airplanes we had four maneuvers, I think, that had to 
be completed in the airplane. That went on for years until 
finally in 1978 we got to landing maneuver, then 
advanced simulation plan, and we still had growing pains. 
In fact, the Royal Aeronautical Society is having a meet- 
ing in November about the extent to which we can trust 
simulation. I think it is a good subject. 

But we needed something to validate what was just 
done in the simulator and I will admit that the selection of 
those maneuvers was somewhat arbitrary. But we did not 
feel we were at the point where we could just say go do an 
ATP check in that simulator, the first one ever. We had 
never qualified one before, and we did not have adequate 
data. We made up for that with the routine that I men- 
tioned this morning. And so we were very conservative 
about it. Maybe we have enough data now — that’s the 
other thing. There were supposed to be data kept and I 
think Greg has some of them. A local office of the FAA 
was to look at how successful we are in that process, at 
how many pilots fail to transition behavior that was 
demonstrated in the simulator to the aircraft. And I don’t 
know how much of that we have, but maybe it’s time to 
ask the question again. I don’t want you to think we are 
stuck with four maneuvers for ever and ever; we are not. 

Regarding the question, why can’t you do an instru- 
ment competency check or instrument rating check, a peti- 
tion would help. We do look at those. Thank you. 

MR. McGOWAN: Warren talked to me before we 
started back. I think we need to clarify that. What Curt 
[Treichel] was mentioning in his presentation is an 
instrument add-on and we did not make application for an 
instrument add-on. What we did make application for was 
certified-flight-instructor-instrument-helicopter in the 
simulator. And that is what the statement that Curt quoted 
in his presentation was about. We were denied that. We 
did make a request for an exemption for certified-flight- 
instructor-instrument-helicopter. One of the reasons we 
were given for the denial of the application was that it had 
never been done in a fixed-wing simulator. So Ed 
[Boothe], you are right, we never have asked for an 
exemption to the helicopter instrument add-on, but we did 
make it for certified-flight-instructor-instrument. 
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7. HELICOPTER SIMULATOR QUALIFICATION 

BRIAN HAMPSON 


I think the one thing that all of us have in common 
here today, and I believe I am quite safe in saying this, is 
that we are all supporters of the view that simulators are 
an acceptable, if not essential, method of training and 
checking aircrew, and that includes helicopter aircrew as 
well. After all, I believe that is the reason we are here this 
week, to discuss how the use of simulation may be 
defined in respect to the training events for which it is 
deemed appropriate, the level of the technology to be 
used, and the criteria that will enable us to get the simula- 
tor approved. From these bases will fall out the design of 
the simulator, and this will in turn be constrained by the 
technology available, which in turn will perhaps lead us to 
modify the use and the criteria baselines. 

So as you will see, to some extent we are going to go 
around in circles. I think this is to be expected at this 
phase of our deliberations, but I believe there are some 
things we all should understand from the outset which 
will help reduce the number of circles we are going to 
describe this week. I base my comments on my experi- 
ence in a similar type of exercise for fixed-wing aircraft in 
which I participated both on the international and national 
levels and also as a result of the knowledge I have of the 
difficulties faced by the simulator manufacturers in build- 
ing and designing a simulator for any aircraft. 

We are fortunate in having a pattern in the fixed-wing 
training and evaluation criteria from which we may start. 
AC 120-40B is a well-debated and currently used docu- 
ment known to most of us. However, as most of you who 
have reviewed the draft AC 120-XX prepared for this 
study can attest, slavish adherence to 40B will not pro- 
duce a good helicopter document. And I am not leveling 
any criticism at the FAA in this area. 

For example, in attempting to get a direct read-across, 
but also by taking note of the unique situation of rotary- 
wing aircraft, the objective tests defined in this draft circu- 
lar total over 800. And that is quite an impractical number 
for any operator to attempt to run, either on an ongoing 
basis or at the time of initial or recurrent inspection. 


I suggest, therefore, that we must begin by using the 
format of 40B, perhaps, but then, by analyzing the impor- 
tant aspects of helicopter training and competency check- 
ing, define the set of objective tests to ensure that the 
device is capable of meeting these training requirements. 
To the objective test must be added, as in 40B and its pre- 
decessors, both functional and subjective tests to ensure 
the necessary realism. We found in the international 
forum that we had to modify our baseline document to 
take into account specific training requirements of other 
national operators. A good example I think was the 
Australians, who have a requirement that their pilots 
demonstrate they can do a rejected takeoff of maximal 
outweight. They naturally said if we are going to check 
somebody on a simulator doing this, we have to ensure 
that the simulator correctly represents a rejected takeoff of 
maximal outweight. I think that gives us a pattern of what 
we should be doing here later in the week. 

If we agree on this, then we must take a closer look at 
these objective, functional, and subjectives tests. Each test 
will consist of a description of the test, a statement of the 
acceptable tolerance between the flown data and the simu- 
lator’s response, that is, the validation, the flight condition 
or conditions under which the test is to be conducted, and 
finally, perhaps, some indication as to the method of 
proving that compliance. For instance, is a time-history 
necessary or will a snapshot do? You may think that this 
is a simple enough matter, but those of us who have been 
involved in the fixed-wing regulatory criteria discussions 
know only too well that the method of actually carrying 
out the test is as important as all the other aspects of that 
test. Again, to give an example, insistence on totally inte- 
grated or end-to-end tests where the control input is 
applied without tolerance, and the output, that is, the 
result of the input on the aircraft, is measured to be accu- 
rate within a given tolerance, is rarely practical within 
currently available technology. 

The greatest problem is the manner in which the air- 
craft data are collected or presented. To again use an 
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example from the fixed-wing area, if the data are obtained 
by plotting, say, the force or displacement of the control 
column and its effect on the surface, and if then a second 
plot is obtained by taking a displacement of the control 
surface and plotting its effect on the aircraft path, it will 
be quite impossible to match the simulator’s results by 
putting an input to the simulator control column and mea- 
suring the effect upon the simulator’s flight path, unless 
the tolerances are generous. That is, it’s quite impossible 
if you are going to apply the same sort of tolerances as 
those that are now specified. To be fair, this is not the 
manner in which rotary-wing checkout data have been 
presented in the past, but the accumulation of tolerances 
on the aircraft owing to differences in manufacture, main- 
tenance, age, ambient conditions, and indeed even the 
data-measuring equipment, would ensure that the end 
result is very much less accurate than that usually permit- 
ted by the defined tolerances of the simulator. What is 
essential is a practical realization of the problems 
involved and the manner in which the data have been pro- 
vided. In the fixed-wing world where aircraft manufactur- 
ers have been collecting this type of data for many years 
now, it has been generally accepted that without spending 
huge sums of money, the currently available data- 
gathering equipment and instrumentation are capable of 
an accuracy that is satisfactory for aircraft certification 
purposes and even for its intelligent use in performing 
checking. However, it is often not accurate enough to 
validate total end-to-end system operation in a simulator. 

To many of us in the study group, the use of the term 
“application of good engineering judgment” is an essential 
part of understanding how a simulator may be said to 
meet the approval criteria. 

For the last few minutes I have addressed a particular 
issue which in part concerns data and how they are used. I 
believe a much more fundamental problem in the simula- 
tion of a helicopter is the amount and the type of the 
design and checkout data which are available. For many 
years now the operators of simulators, the bodies repre- 
senting them, and the manufacturers of simulators for 
fixed-wing aircraft have been trying to define a minimum 
standard for the data that are to be supplied for these pur- 
poses. The third edition of the IATA Data Document was 
published in 1990. It is the result of several years of effort 
by people very experienced in the manufacture, testing, 
and use of fixed-wing simulators. And it enabled some 
progress to be made in defining acceptable criteria for the 
fixed-wing simulator. Few, if any, rotary-wing aircraft 
manufacturers come anywhere close to meeting similar 


standards which may have been defined, but I do note 
with pleasure that the data analysis document provided by 
Augusta for the A- 109 simulator indicates that they may 
be an exception to this criticism. Cost is only one of the 
reasons given for the failings of the aircraft manufactur- 
ers. Because helicopter simulator approvals have not been 
such a prominent item as the fixed-wing ones until now, it 
is easy to understand why the scope and accuracy of heli- 
copter data packages have been inferior to even the 
mediocre ftxed-wing packages. What must be accepted is 
that any move toward defining higher criteria for evalua- 
tion, testing, and improvement in the training obtained 
from helicopter simulators will require an order of magni- 
tude of improvement in the data being supplied. 

It has been said that the average helicopter data pack- 
age is the equivalent now of what the fixed-wing data 
package was 15 years ago; some would even say 20 years 
ago. A continuation of this approach is not commensurate 
with the building and evaluation by a regulatory authority 
of a helicopter simulator equivalent to even Phase 2, 

Level C standards. The success of the FAA’s Advanced 
Simulation Plan for fixed-wing aircraft is well-known, but 
I hazard a guess that it would not have been so effective in 
reaching its goal of zero flight-time training were it not 
for the work put in by the IATA Flight Simulation 
Committee in defining the required level of data. 

Unfortunately, IATA has not, to my knowledge, con- 
vened a committee to set up similar data standards for 
rotary-wing aircraft, although the “Aircraft Data and Sup- 
port Requirements Document for Aircrew Training 
Devices,” produced in 1988 by the Naval Training Sys- 
tems Center, does address some of the issues, including 
those of the data requirement for rotor-map and blade- 
element models. This general deficiency must, in my 
view, be rectified as part of the exercise on which we are 
about to embark. 

I would now like to give some examples of the areas 
that I believe are insufficiently addressed in current data 
packages. First, helicopter data packages frequently do 
not include any models at all and, hence, no proof-of- 
match document. It is left to the simulator manufacturer to 
design these. This is not the most efficient way of solving 
this problem and may lead to greater variations in the 
simulation of one manufacturer’s product and another’s 
than is now the case for fixed-wing aircraft. 

Second, the inherent instability of the helicopter is 
known to all those who attempt to fly it. Most modern 
types have stability augmentation systems which are used 
full time. The data covering operations without the 
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systems in use are sparse, yet this is an area of prime 
importance to training. 

Third, vibration is likewise a fact of life in heli- 
copters. Indeed, operation is frequently constrained by the 
need to avoid and suppress critical vibration frequencies. 
But the data provided on those vibrations are rarely 
comprehensive. 

Fourth, flying operations of helicopters, especially 
operations close to the ground or to the surface of water, 
require accurate modeling of the downwash and knowl- 
edge of the prevailing conditions. This in turn requires 
very accurate recording of ambient conditions with a 
larger number of parameters being recorded at a higher 
frequency than is now common. 

Based on the analysis document provided by 
Augusta, which I mentioned previously, it would seem 
that they at least accept that none of these difficulties is 
insurmountable. It is this recognition of the need for a 
level of data commensurate with modern techniques and 
technology that is required. However, it is unfortunate that 
this is in exact contravention to the view I have heard 
expressed here today and in many other forums of this 
sort, that is, the matter of reducing cost. 

You cannot get this level of data and this level of 
simulation without spending a lot of money. Some of the 
issues of cost have been raised this morning and indeed in 
the last paper. Something like 60% of the cost of a fixed- 
wing Phase 3 simulator goes not to the building of the 
simulator but to providing the data package and the air- 
craft parts and avionics that go into that simulator. And 
that is a problem I don’t see any way of overcoming sim- 
ply, if we are to provide the degree of simulation that is 
expected of a Phase 3 device. So we are faced with a 
$12 million or more bill for a Phase 3 helicopter simula- 
tor. That’s not to say we cannot produce Phase 2, Phase 1 , 
or even flight-training devices at less cost. It is a matter 
of, as the last speaker said, looking at the return you are 
going to get. 

To put this in context, we heard some figures men- 
tioned earlier about the price of operating a simulator. If 
you take the cost of operating an airplane and compare it 
with the cost of operating a simulator, it is a ratio of about 
10-to-l. To give you the top-end example, if you look at 
the cost of ownership of a 747 simulator on a per-hour fly- 
ing basis, which includes the amortization, the cost of the 
device, and the building in which it is housed and all the 
utilities it needs, you come up with an operating cost of 
about $450 to $500 an hour. If you go to the airplane and 
use the same criteria, that is, cost of purchasing the air- 


plane, cost of the crew, the increased cost of maintenance 
caused by the effects of the repeated landing cycles on the 
engine, the wheels, the brakes, and the undercarriage, and 
the additional insurance costs, it is an accepted fact in the 
fixed-wing world that the cost of operating a Boeing 747 
for 1 hour of training is $16,000. As I say, that is a top- 
end one. On an average we are talking about a cost ratio 
of 10-to-l in operating the airplane over the simulator. 

Now, obviously for somebody who is only operating 
two or three airplanes, they have a problem. And I think 
we need to get the thing into proper perspective. Unless 
you think I am being unduly pessimistic, let me hasten to 
reassure you, we believe the manufacturers have proved 
their ability to provide highly accurate simulations of 
some of the most advanced helicopters currently in opera- 
tion. These have, almost without exception, been built as 
military programs and have been successful because addi- 
tional data have been provided through simulator data- 
gathering exercises on the aircraft and by a large invest- 
ment of pilot and design engineer time in tuning the mod- 
els or final results to meet the objective assessments of the 
pilots. Such expensive methods will probably not be 
acceptable to the average civil helicopter operator, who in 
most cases will not have the resources of the military nor 
of the large fixed-wing aircraft operators. 

Yet despite holding this view, I can also add that 
because of the special circumstances surrounding some of 
the training problems for helicopters, there may be no 
other alternatives. For example, in the relatively high 
speeds encountered even in large transport airplanes, the 
human vestibular system is easily fooled into believing 
that the onset cues or short-term changes produced by the 
motion platform are being sustained. With the heli- 
copter’s low-speed operations, the combination of visual 
cuing and motion cuing may not have such a good effect. 

I believe the motion cues become more important in a 
relative sense, because the rate-of-change cues from the 
visual scene at low speeds are small. Not all of my col- 
leagues will agree with this point of view and that, in 
itself, is sufficient reason for raising the subject now. 

The adoption of an advisory circular to control the 
evaluation and approval of helicopter simulators is 
specifically designed to remove all but the smallest 
amount of subjectivity and to permit recurrent inspections 
to be carried out from an objective baseline. The first of 
these aims may be impossible to obtain until better data 
are available. And the the second may prove impractical 
and probably unacceptable to the regulatory authorities. 
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My point in raising the issues I have addressed in this 
presentation is to warn against falling into the trap of 
thinking that all that is necessary as an outcome of this 
workshop and the ensuing working group for the advisory 
circular is the definition of the training events and the 
evaluation criteria. Both of these items are essential to the 
task at hand, but they will be negated unless we also 
address the problem of the data and how they are to be 
used. 

It has taken some 12 years to reach that conclusion in 
the fixed-wing world. I submit we cannot afford to give 
the same amount of time to helicopter simulators. Thank 
you very much. 

MR. CARVER: Brian, is not today’s problem with 
helicopter data collection and the construction of the doc- 
ument similar to the one which has been sent out here by 
Ed [Boothe] and his compatriots and the same situation 


we were in with fixed-wing where actually we have all 
managed working together to achieve everything that is 
required. Are we not, by using your suggestion, choking 
off development for the future? 

MR. HAMPSON: I think there is some value in what 
you said, Paddy. My only comment really on what you 
have said is that we have a different group of players here. 
And what I was trying to do in my paper, and I am sure 
you support the view, is to try and read across some of the 
experience we got in the fixed-wing world so we do not 
have to spend 12 years in the helicopter case, as we did in 
reaching the conclusion we reached in the fixed-wing 
case. And I certainly would not want to choke off any- 
thing, but there are some exercises, were we to go back 
12 years, in the fixed-wing case that we would almost cer- 
tainly do differently. I do not think any of us who have 
been involved in it would disagree with that. 
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BOAC/British Airways. He has been a member of the FAA working groups on 
AC 120-40B and 120-45A and on CAP 453 in the United Kingdom. He is also 
chairman of the RAeS conference currently seeking to establish international criteria 
for flight simulators. 
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8. HELICOPTER SIMULATION: MAKING IT WORK 

BARRY PAYNE 


The benefits of flight simulation are well docu- 
mented. The evidence is in daily practice throughout the 
world, but so far is confined mainly to fixed-wing avia- 
tion. Yet, the opportunities for improved training and 
checking using helicopter simulators are greater than for 
airplane pilot training. For example, simulators facilitate 
training environments conducive to the development of 
pilot decision-making, situational awareness, and cockpit 
management, all skills that are essential to a reduction in 
human-error accidents. 

Accident data compiled from New Zealand’s Air 
Transport Division mirrors data and reports from the 
NTSB, the FAA, the U.S. Army, and the Canadian heli- 
copter operators. These data indicate that most helicopter 
accidents involve complacency or lack of training in how 
to handle the “chain of errors” that generally results in an 
accident. New Zealand studies confirm that most heli- 
copter accidents in that country are also caused by pilot 
error, that these are not confined to any group of experi- 
ence levels, and that 65% of the causes listed are not spe- 
cific to the helicopter type. It is also worthy of note that 
helicopter accident rates have not seen significant 
improvements even though the machine’s reliability has 
improved. 

Studies from around the globe readily confirm what 
helicopter operators already know — the rate of accidents 
is too high and human error is the leading factor in avia- 
tion mishaps involving professional pilots. 

Eighty percent of the world’s helicopters are single- 
engine types operating almost exclusively in VMC and 
performing everything other than a flight from one airfield 
to another. Today’s helicopter pilots operate in environ- 
ments that require a wide range of skills that were not 
likely to have been addressed in traditional training. Most 
operators are conscious of this and do their level best to 
manage risks. However, for a great many this task has its 
own special difficulties. 

For example, how effective can you be when the 
operation utilizes 28 helicopters comprising six different 


types flown by 86 pilots of various nationalities all work- 
ing in a foreign country and scheduled on flexible tours to 
perform a wide range of tasks in an environment that 
could involve sea-level jungle operations or mountains 
typically at 9,000 to 12,000 feet with temperatures of ISA 
+20. In these circumstances, for helicopters operators 
based in Papua, New Guinea, training and checking have 
their own special problems. 

Likewise, a typical operator in New Zealand may 
operate two helicopters, both different types. These could 
be flown by two full-time and two part-time pilots. Any 
pilot may be expected to spray potato crops before break- 
fast, sling drilling material and supplies late morning, 
undertake a corporate mission in the early afternoon, and 
be called upon to consider a medivac after dark. A small 
Australian operator with one helicopter type may be sup- 
ported by two casual pilots who also supply their services 
to at least three other operators, and in the course of their 
duties fly several different helicopter types on a variety of 
tasks, each with its own peculiar standards. 

Although the examples used here are focused on the 
southwest Pacific area they illustrate a point that is com- 
mon to a great deal of the international helicopter frater- 
nity. That is, the use and variety of operational tasks 
expected from a helicopter are many times more varied 
and considerably more complex than those involving air- 
planes. Additionally, the commercial and economic reality 
of our industry will continue to ensure that even more 
innovative ways will be found to increase helicopter uti- 
lization. The risk-management difficulties faced by the 
average helicopter operator therefore can be quite com- 
plex. This task is often further exacerbated when the best 
solutions must also confirm with a regulatory require- 
ment, the roots of which may have been specifically 
designed for an IFR airplane operation between airports. 

Any pilot involved in training and checking commer- 
cial helicopter pilots can forecast with relative accuracy 
the types and circumstances of accidents that will occur 
within various operational roles. For example, it can be 
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said with assurance that within the month, somewhere in 
Papau, New Guinea, a pilot with more than 1,500 hours 
flight time and the benefits of recent sling-loading experi- 
ence will be involved in an accident as the result of pilot 
error while sling loading. The circumstances will not be 
new. It may be the result of a skid having caught in a net 
while lifting off, or a rotor-strike while attempting to 
recover from a downwind approach without releasing the 
load. Whatever the cause, it will not be a new one, but a 
well-tried one repeated. In New Zealand this winter we 
can again expect a helicopter pilot to enter a cloud while 
trying to remain visual and as a result lose control and 
crash. The human-error accident, unfortunately, is the eas- 
iest to predict. 

A study of New Zealand helicopter accidents from 
1980 through 1989 showed that fewer than 10% of the 
human-error causes could be considered peculiar to the 
helicopter type involved. Very few accidents involving 
helicopters have a cause limited to only one specific 
manufactured type. 

The reduction of human error is the most fertile area 
for an improvement in our helicopter accident rates. Uni- 
versally the helicopter accident rate is managed by means 
of training and checking programs, the minimum require- 
ments of which are usually determined by civil aviation 
regulations or rules. However, it is the quality and content 
of this training that will determine if the helicopter acci- 
dent rates remain constant or are reduced. 

Since there are obviously far more applications for 
commercial helicopters than for airplanes, there would 
seem to be a requirement for a greater diversity of skills 
among helicopter pilots. This strongly suggests a greater 
need for quality recurrent training with an emphasis on 
the occurring factors as evidenced in accident data. It is in 
this role that the helicopter simulator has its greatest 
future. 

The airplane simulator has proven the benefits of 
simulation in imparting quality training. A study by 
United Airlines concluded that training in the flight simu- 
lator was 150% more effective than training in the actual 
aircraft. Simulator development for the airplane industry 
has been driven by cost benefits and regulatory com- 
pliance. Identical factors would also power a helicopter 
simulator industry. Cost-effective simulation, together 
with rules that would recognize training credits, would be 
sufficient for many operators to move their training and 
checking in the direction of helicopter flight simulation. 
The principal element involved is that the needs of a typi- 


cal helicopter operator are very different from those of an 
airplane operator. 

The use of helicopter simulation as a pilot recurrent- 
training tool has the potential to reduce accident rates, 
which has not, so far, been achieved using currently 
applied methods. For example, a sling-load training exer- 
cise with a pilot who incorrectly judges the wind direction 
and attempts a downwind approach could not be contin- 
ued beyond a very early stage, for the risk to machine and 
occupants would be too great. In the aircraft, the training 
captain may establish the gravity of a given situation; 
however, the pilot concerned may not recognize a similar 
situation in the future because it was not prudent to repeat 
the exercise. The same exercise conducted in a simulator 
could be continued to conclusion and then repeated to 
illustrate the cues that could be used to recognize a similar 
situation again. Such training methods usefully demon- 
strate the benefits of procedures, decision points, etc. 

Like a great many of the skills a helicopter pilot must 
maintain, sling-load training is not entirely helicopter- 
type-specific. The same background skills and experi- 
ences are applied to all sling loading regardless of what 
helicopter type is being operated. The same analogy can 
be made for many helicopter tasks ranging from hovering 
to mountain flying. To be effective, helicopter simulation 
must meet the broad needs of the 80%, mostly single- 
engine, VFR-only segment of our industry. 

Based on our own experience, the evolution of simu- 
lation software, hardware, and visual systems can cur- 
rently provide realistic and cost-effective helicopter simu- 
lation. Present technology can field a fixed-base cockpit, 
equipped with 150° day/night visuals and capable of 
mountain flying, sling loading, elevated heliports, etc. 
Such a device can be operated at costs that equate favor- 
ably with light turbine helicopters. Results can verify 
effectiveness. It is a fact that right now helicopter simula- 
tion has the capability of providing operators with the best 
risk-management tool available. 

The conflict occurs when a definition of helicopter 
simulation is required in order to satisfy present rules and 
regulations. Immediately, comparisons are made with air- 
plane simulators built to satisfy regulatory requirements 
for type transition, recurrent, and route training and 
checking. Although such requirements will fulfill the 
needs for a segment of the helicopter industry they fall 
wide of the mark when compared with the majority needs. 

The establishment of our helicopter flight simulator 
in New Zealand first highlighted some of the difficulties 
that have yet to be resolved. In the absence of local policy 
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and relevant regulations, our air Transport Division 
looked to the FAA for assistance. As a consequence, we 
can foresee the very real danger that specifications and 
requirements applicable to the airline industry will be 
applied to helicopter simulation. Such an approach to rule 
making would no doubt keep helicopters simulators well 
out of reach of those 80% who need them. 

By way of example. New Zealand’s aircraft civil 
register lists approximately 330 helicopters (Australia has 
around 4,000). Typically, these constitute a mixed fleet of 
various types and models engaged in a wide variety of 
operations. As a comparison, the combined value of New 
Zealand’s helicopter fleet would not exceed the value of 
two Boeing 747 airliners. Advanced training and checking 
technology translates into very costly equipment which 
has to be justified against relative values. 

The answer may well lie within the significant 
research work that has been undertaken since the advent 
of modem flight simulation. Sufficient verification by 
authorities such as Alfred T. Lee and Paul W. Caro has 
removed the blurred distinctions that exist between train- 
ing technology and flight-simulation technology. To pro- 
vide characteristics of the helicopter that do not support 
that training objective is to increase the cost of the system 
for cosmetic rather than training purposes. Acceptance of 
such criteria will be fundamental to ensuring cost- 
effective helicopter flight simulation. 

New helicopter-simulator criteria are vital and they 
should be in place now. A great many of the skills 
required by helicopter pilots are not type-specific and 
indeed could, for that matter, be accomplished in a 
generic simulator. Hovering, sling-loading, confined-area 
landings, mountain flying techniques — the list goes on. 
When using a simulator to check a pilot’s emergency pro- 
cedures in the event of an engine failure while carrying a 
sling load, the position of the cargo release becomes a 
mere detail if the pilot did not even consider releasing the 
load. 

There are many important skills that contribute to 
safe helicopter flight. They apply to all pilots regardless of 
the type of aircraft or style of operation. Their relative 
importance, however, may be different for each crew 
member and operation. These are skills that are highly 
suited to be learned and practiced in the course of simula- 
tor training and checking exercises. They are: 

1 . Cockpit distractions 

2. Stress management 

3. Use and function of checklists 

4. Communication skills 


5. Workload assessment and time management 

6. Decision-making and judgment 

7. Management of flight resources 

8. Managing people 

9. Flight planning and progress monitoring 

10. Pattern (chain of events) recognition 

The state-of-the-art visual systems, such as the IVEX 
VTS 1000, can provide realistic cueing sufficient to con- 
duct simulated day-time operations including hovering 
exercises. When such visual systems are integrated with a 
fixed-base cockpit exhibiting genuine helicopter charac- 
teristics, there begins to emerge a practical training tool 
fully capable of influencing the unfavorable accident 
statistics generated by the helicopter industry. 

Although the practical benefits and training effec- 
tiveness of helicopter simulators can be argued, wide- 
spread acceptance of such devices by operators will 
largely depend on the results of rule makers and the 
training and checking credits available to offset the use of 
actual aircraft instead. 

MR. LOMBARDO: Several times today I have heard 
this recurrent theme about the procedures, that it is not so 
important that the simulator be exact in terms of hard- 
ware. There is a piece of research that just came out, in 
the most recent issue of the Human Factors Journal, and I 
will quote it in my paper tomorrow in the low-cost ses- 
sion. But for the benefit those of you who cannot attend 
that session, a researcher has taken a group, split them in 
half, had one group learn to deal with the conceptualiza- 
tion of a piece of equipment, and then they went on to try 
and do the task on that equipment. Another group learned 
to do the procedure, but on a piece of equipment that 
wasn’t the same as that used for the final tasks. Guess 
who won? The group that practiced the procedure won 
over the group that was familiar with the hardware. They 
were more readily able to adapt a known procedure to 
another piece of hardware than they were just to shift the 
concept of how something works. 

So that recurrent here is a very, very strong theme. 
That is what I think we are looking for — the procedure. 

MR. PAYNE: I agree with you. We can illustrate the 
point that every year somebody ends up autorotating a 
helicopter and putting it on the ground when it was per- 
fectly serviceable to begin with. That is, it was perfectly 
serviceable up to the minute that it touched down. What 
the pilot saw and reacted to was what he thought was an 
engine failure. All his training taught him to do autorota- 
tion, touch-down autorotation. But the opportunity doesn’t 
occur often enough to break down bit by bit what is 
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actually happening. So every year, your statistics, our 
statistics, show that if anyone who has a gauge failure and 
who doesn’t pick that up as a gauge failure reacts to an 
engine failure, rolls the throttle back like they do in prac- 
tice every time and carry on to the ground and usually 
muck it up. 

And a simulator can help identify that. It will cer- 
tainly provide the training in identifying the problem and, 
again, it can be a turbine simulator. It does not have to be 
one particular type. 

MR. WALKER: I seem to see a difference in opinion 
about the requirement for ground contact maneuvers 
between you and the PHI paper [Gerald Golden, Petro- 
leum Helicopters, Inc.]. Is that true? 

MR. PAYNE: Well, I can understand any operator 
who says I don’t want my equipment being smashed onto 
the ground. There are even experienced instructors who 
may not have the judgment, the continued day-after-day 
judgmental skills to ensure that an operator’s very valu- 
able equipment can exit a touchdown autorotation in a 
100% serviceable condition. And I can understand any 
operator who says I don’t want my equipment being sub- 
jected to that risk for training. That is a reality of life. So 
it does not obscure the fact that touchdown autorotations, 

I believe, are a very necessary part of training. 


Our simulator does a pretty good simulation of a 
touchdown autorotation, although the last couple of feet 
are not all that realistic. But it becomes a lot more realistic 
when winding on the throttle at 100 feet and recovering 
with a flow through. What’s more, you don’t have to fly 
the circuit in between to reposition the helicopter. Again, 
you start from 2,000 feet. You can do repetition autorota- 
tions that have a lot of training value. My opinion is that 
the autorotation is a skill that the pilot must have, and 
maybe a simulator is a way of providing it with less risk. 

MR. KATZ: This is a combination of a comment and 
a question. I very much appreciate and like what was said 
here about the skill being, generic — I think the term was 
not used, but this is what it meant. Many of the skills are 
not type-specific. And therefore adherence, fidelity to a 
particular type, is not essential to get the training benefit. 
And I would like to throw out the suggestion that maybe 
you don’t really have to adhere to any particular type, and 
maybe the most cost-effective way to reap training bene- 
fits for generic skills is in a generic simulator which may 
be a physically correct helicopter, which nevertheless 
does not correspond to any actual type. 

MR. PAYNE: Thank you, I agree with you. And it 
certainly makes the collection of data to produce the 
model much easier. Thank you. 



Barry W. Payne has worked extensively in the field of recurrent pilot training and 
human factors. He qualified as an aircraft engineer, airplane pilot, and helicopter 
pilot while a member of the Royal New Zealand Air Force. Following his military 
service, Mr. Payne worked in various general aviation roles throughout the southwest 
Pacific and Asia. He is a licensed aircraft engineer, an A-category instructor pilot 
with instrument rating, and has more than 10,000 flight hours. His company. Avia- 
tion Network (NZ) Limited, operates a Bell 205/UH-1H simulator which is used to 
train military and civilian pilots. 
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9. HELICOPTER TRAINING SIMULATORS: KEY MARKET FACTORS 

JOHN MCINTOSH 


The training services and training equipment industry 
has been working in partnership with NASA and the FAA 
to constantly improve the ability of people in the air 
transportation network to perform their missions. This 
workshop is but another step in bringing technology and 
performance standards to bear on the training of heli- 
copter crewmen in the civil sector. Your review of and 
contributions to the draft FAA Advisory Circular for 
Helicopter Simulator Qualification can significantly affect 
the quality and cost of pilot training for years to come. 

I don’t know whose idea it was when the FAA came 
out with its first Advisory Circular for fixed-wing simula- 
tor qualifications. You all remember “Appendix H.” 
Whoever it was, ought to get a medal! That development 
established standards that have saved uncountable mil- 
lions of dollars, provided a basis for vastly improved 
training, and provided a model copied around the world 
and by our own military in some procurements. Extending 
that precedent to vertical-lift aircraft is consistent with the 
advances in helicopter simulation technology and with the 
future demands on helicopter pilot training. 

I wish to present an analysis of that future demand 
and to discuss some of the factors that will influence the 
market for helicopters and simulators. I will also touch 
briefly on other vertical-lift market offerings, including 
tilt-rotor and tilt-wing aircraft. 

My sources include interviews with major helicopter 
and vertical-lift aircraft manufacturers, NASA studies, 
interviews with industry providers of training services and 
equipment, trade journals, and other published data on 
aircraft operating costs. 

There are a number of factors that will influence the 
future demand for helicopter simulators. Chief among 
these will, of course, be the demand for civil helicopter 
aircraft and the types of the units sold and their missions 
(fig- !)• 

The forecast shown in figure 1 covers the period 
1991-2000. Although the delivery of civil helicopters 
looks relatively flat through this decade, notice the trend 


toward light twins and intermediate helicopters. Light 
twins are defined as aircraft under 6,000 lb, and interme- 
diates comprise the range of 6,000 to 15,000 lb. Most of 
the simulation equipment built to date has been for aircraft 
in these two categories. 

The delivery of 5,330 aircraft in this decade will 
roughly break out at one-third domestic and two-thirds 
worldwide, with the hot markets being in densely popu- 
lated areas such as Japan, the rest of the Pacific Rim, and 
Europe. There are some who feel that a critical juncture 
will be encountered in the 1994-1995 period, one that will 
be brought on by basic decisions on how to handle air- 
transport systems overloads. One scenario, which I will 
discuss later, could distort the delivery picture radically 
and impose heavier demands for simulator training in the 
last half of the decade. With that, let’s take a look at some 
of the forces that shape the demand in the helicopter mar- 
ket (table 1). 

There are several factors that are favorable to the 
helicopter market. The export business remains strong and 
is growing in densely populated areas. These are areas 
where all means of surface and air transport are becoming 
overburdened. Additional interest for emergency medical 
services and public sector helicopter utilization is also 
related to population growth, required response times, and 
available capital. 

Conversely, the lack of infrastructure rather than 
overtaxed, developed infrastructure, is going to influence 
growing helicopter demand in Third World nations. There 
is no question that a possible up side scenario to the fore- 
cast shown in table 1 does exist and that it could kick in in 
the mid-1990s. 

While development of the economies of Eastern 
Europe will provide market expansion, the supply side 
will be developed also the civil competition from the 
U.S.S.R. and other sources. Eurocopter could be a syner- 
gistic giant compared with the founding partners of 
Aerospatiale and MBB. The infrastructure for vertical lift 
is also growing along with helicopter demand; it includes 
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Figure 1. Ten-year civil helicopter forecast: 5,300 aircraft by weight class. 
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Table 1 . Market forces 


Positive forces 

1. Export possibilities are growing 

High-density population areas 
Third World development 
Eastern Europe trading 

2. Vertical lift infrastructure is expanding 

3. New vertical-lift technologies may provide explosive growth to passenger and package express possibilities 

4. More reliable rotorcraft with reduced operating costs 
Negative forces 

1. Environmental concerns 

2. Safety and public image 

3. Availability of capital investment 

4. Competing technologies 


pads, facilities, and, now, vertiports. Vertiports like the 
one planned for downtown Dallas can handle transitional 
vehicles such as tilt rotor and tilt wing, as well as 
helicopters. 

If the newer “tilt” technologies are successful in 
penetrating the public sector passenger and cargo markets, 
and if the air-space regulations and infrastructure are 
properly developed concurrently, then there will be a fall 
out of additional helicopter demand capable of exploiting 
the same facilities and the same regulatory climate. For 
helicopters to position themselves for this market share 
the good work now being done to reduce seat-mile costs 
and to improve reliability, perceived safety, and environ- 
mental compatibility must be continued (table 1). 

It may have been all right for President Reagan to 
stand near his helicopter with his hand cupped over his ear 
saying “Sorry Sam I can’t hear your question,” but most 
folks do not take kindly to noisy machines belching 
exhaust in their neighborhoods. It gets particularly alarm- 
ing when one of those machines makes an emergency 
autorotation down into a busy intersection. The public 
will have to be convinced that helicopter use can be 
expanded in a safe, environmentally compatible manner 
before they will vote the funds for helicopter purchases by 
police or for medical services or facility construction. 
Given the right technology, they might accept vertical-lift 
aircraft, at least as much as they do fixed-wing aircraft. 

Capital is hard to find right now and it will continue 
to be so until debt loads are relieved and GNP’s are on the 
rise again. This isn’t the financial climate for getting a 
loan to build a beer hall in Baghdad, but investments that 
make sense, show a return, and are in the best interest of 
government, industry, and the public can still be managed. 


Planning, combined with technology, can benefit vertical 
lift. 

There will be competition for the funds and project 
support. Take the Boston-New York- Washington corridor 
for example. Reliever airports, additional runways, heli- 
pads, magnetic rail systems, and bullet trains will all be 
competing for the same pot of money. 

Aside from all the light singles driving the training 
and private use numbers, the market continues to be 
driven by the working needs of the oil industry (table 2). 
By and large, the helicopter remains a working tool whose 
price is justified by the revenue returned for the task to be 
performed. Today its sales and use are still affected by a 
poor public image as a vehicle for general transportation. 
That image could change in the 1990s, but several factors 
will have to be overcome (table 3). 

The seat-mile costs of helicopters are about twice 
those of regional fixed-wing aircraft, and the “tilt” tech- 
nologies will bring that disparity down from 2 to 1 to 
about 1.2 to 1.4 to 1. Obviously other economic issues 
remain to be dealt with. Progress toward resolution of 
some of them is promised by the cost model of a complete 
door-to-door transportation scenario that applies a cost 
factor to the total time saved, as the air-traffic control 
system and facilities are further tuned to city-center-to- 
city-center operations. As the infrastructure grows to offer 
more possibilities, the economic model will improve as 
well. 

Bear in mind, however, that other competitors for the 
traveler’s dollars will not be standing still as constants in 
the economic model. They will be moving hard to capture 
public and private capital. 
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Table 2. Civil helicopter market segments 


Segments 

Training 

Petroleum and industrial 
Public service 

Emergency medical services 
Executive/corporate 
Passenger 
Comments 

7,400 helicopters now flying in United States; main driving force: petroleum industry 
Public service, including law enforcement, operates 1,400 units in 335 agencies 
174 emergency medical services programs operate 231 units 


Table 3. Vertical-lift passenger traffic market: factors opposing 


1 . Vertical-lift technology must overcome: 

Noise and other environmental concerns 
Seat-mile cost disadvantage 

Lack of dependable IFR operations in icing conditions 
Negative public image of safety and reliability 

2. Air-traffic control systems must be changed to accommodate higher volume vertical-lift IFR traffic in vertiport 
infrastructure 

3. Other modes of transportation are competing for private and public capital: 

Reliever airports 

Bullet trains 


Today’s congestion makes the case for civil tilt-rotor 
and tilt-wing research (table 4). The air-travel delays 
today at those 2 1 airports are estimated to cause a 
$5 billion annual loss. By the year 2000, this grows to 
50 airports with this magnitude of delays. Eight-four mil- 
lion dollars, the rough cost of an extra runway, is enough 
money for several helipads and for the tilt-rotor aircraft to 
use them. That structure, if it happens, will pump heli- 
copter sales as well. It could very well be that the first 
working example of this will occur in the densely popu- 
lated Japanese travel sector. The industry study team, 
studying tilt-rotor missions for NASA, reported that a 
single new airport would cost $4 billion to $6 billion. For 
half that cost, they estimated that an entire network of 
12 urban vertiports could be built along with 165 40-seat 
tilt-rotor aircraft. 

High fidelity and cost-effective training will continue 
to gain in importance in the vertical-lift market we have 
been looking at. 


You all know that simulator fidelity isn’t legislated or 
wished into being. The right data must be modeled in the 
right way and implemented on equipment capable of exe- 
cuting the model and cues in real time. 

The forces acting on the vertical-lift market, which 
we have reviewed today, will create a continuing training 
demand. The trends indicate that the training will continue 
to shift toward simulation equipment and that the training 
will be provided by full-service training companies. Some 
key people are expecting a significant increase in 
simulator-base training demand in the 1995-1997 period. 

As was true in the fixed-wing experience, the accep- 
tance and use of simulator-based training will be influ- 
enced by simulator fidelity, economic advantage, and a 
regulatory environment that permits credits for the train- 
ing given. Helicopter simulation fidelity (table 5) is more 
difficult to achieve, in some ways, than is fixed-wing 
fidelity. 

To begin with, rotors present a unique problem, 
given their flexibility and varying angles of attack. The 
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Table 4. Vertical-lift passenger traffic market: factors favoring 


1. Twenty-one airports now have delays of 20,000 hours annually. Predicted: by 1997, 33; by 1998, 41 

2. By the year 2000, prediction is for a 32% increase in jet transports and a 74% increase in passengers 

3. Half of today’s commercial fleet is used for flight segments of less than 500 miles 

4. Situation worse in Europe and Japan 


Table 5. Simulator training: fidelity 


1 . 

Aircraft data and data collection 

2. 

Modeling techniques 

3 

Visual and motion cues 

4. 

Standards for performance 

5. 

Training program design 

blade-element solution offers an improvement over the 
process of tailoring a rotor-map-based design. Its use. 


however, requires model solution speeds unheard of in 
fixed-wing simulator configurations. 

Unfortunately, the modeling and data problems don’t 
end with the rotor. Fuselage aerodynamic data are diffi- 
cult to gather and to document for slow forward air 
speeds, in wind, and in hover. Today, engineers have to 
“twiddle” with induced velocities, and there is a need for 
more data for translational lift. In slow-speed regimes, 
more and more and more resolution is required. Thirty- 
two-bit, floating-point computers will be needed. 

Helicopter motion and visual cues are more compli- 
cated than they are for fixed-wing aircraft. Field of view 
is greater, with down-look angles that are important. Also 
important in helicopter training is the fidelity of onset and 
vibration cues. 

Perhaps the biggest technical problem is the unavail- 
ability of binocular vision in the visual system. The low 
approach to the ground of a fixed-wing aircraft is fast 
enough to reduce the effects of this lack of height cue, but 
a helicopter hover to landing or autorotation is quite 
another matter. Confined-area vertical cues help, but the 
fidelity problem still exists. 

We should all remember that a qualified simulator is 
still but a tool in a pilot-training program. The program 
itself must be designed, by the certificate holder or 
training-services company, to a high degree of quality and 
cost effectiveness. 

Let’s see if we can quantify some of the costs 
(table 6). I have made these reasonable assumptions as a 
basis for comparing simulator training costs with the 


alternative of training in the helicopter. These costs 
(fig. 2) do not include any adjustment for the fact that 
simulator training hours can be more highly concentrated 
and can include training in recovery from a number of 
emergency or otherwise abnormal situations. Certainly 
there is more realism in the real-world environment, but 
there is more safety in the simulator. 

Summing up key simulator market factors, I would 
conclude that fidelity is strong but with some key issues 
revolving around data collection and visual simulation 
remaining to be solved (table 7). The cost equation is 
practical and the demand is reasonably strong with 
mid-decade factors coming into play that could capture 
the attention of manufacturers. 


Table 6. Simulator training: cost/hour assumptions 


Light twin with simulator cost twice that of actual 
aircraft 

1200 flight hours and 3500 simulator hours 

Depreciation over 10 years 

Crew compensation and insurance not included 


Table 7. Helicopter simulators: key 
market factors 


1 . Fidelity of simulator training 

Data and models 
Equipment technology 
Training programs 

2. Cost of simulator training 

Versus training in aircraft 
Trend 

3. Training demand 

Vertical-lift market in the 1990s 
Helicopter demand factors 
Trends 
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Today there are only a few civil helicopter simulators 
that would fall into a classification covered by the FAA 
draft Advisory Circular that is under consideration at this 
workshop (table 8). 

Table 8. Civil simulators 


Bell 222 
Bell 212/412 
Sikorsky S76A 
Sikorsky S76B/A 
Boeing Vertol 234 
Aerospatiale 332L 
Sikorsky S61N 


The forecast for new simulators in this decade is 
shown in table 9. 

Table 9. Simulator forecast: 1990-2000 


1 . Light singles and twins, less than 6,000 lb 4 


2. Intermediate, 6,000 to 15,000 lb 3 

3. Medium to heavy, more than 15,000 lb 1 

4. Other vertical-lift 

Tilt rotor 1 

Tilt wing 1 

Total 10 


Whether these predicted buys are actually made will 
depend on all the market forces we have discussed today, 
not the least of which is the final form and implementa- 
tion of the FAA rules for simulator qualification. 
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Figure 2. Simulator training: cost/hour. 
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10. TRAINING EFFECTIVENESS ASSESSMENT: METHODOLOGICAL ^ 

PROBLEMS AND ISSUES 


KENNETH D. CROSS 


If any one of you has talked with a person who has 
recently examined the literature on helicopter simulator 
training effectiveness, I’ll bet you dollars to donuts that 
they were positively shocked by the small amount of 
research that has addressed this important topic. The per- 
sons I have talked with ask me, “How can it be that the 
military has invested enormous sums in helicopter simula- 
tors without having solid empirical data on how effective 
they are and how they should and should not be used?” 

Although there is a host of reasons for the lack of 
data on helicopter simulator effectiveness, it is my con- 
tention that one of the most important is the lack of an 
evaluation methodology that yields comprehensive and 
valid training-effectiveness data in a timely manner at an 
affordable cost. Accordingly, my comments today are 
aimed at identifying some of the methodological problems 
encountered in assessing the training effectiveness of 
helicopter simulators and some of the issues that must be 
addressed in developing solutions to these problems. 

Before proceeding, it is important to acknowledge 
that my comments reflect the perspective of a behavioral 
sciences researcher (table 1). It is also important to 
acknowledge that my views have been greatly influenced, 
and perhaps biased, by my experience in considering the 
training needs and problems of Army aviators. I have 
attempted to make all of my comments relevant to civilian 
aviation, but I cannot promise that I have been completely 
successful. 

Because time is short, I have limited the focus of my 
comments. The methods I discuss are ones that I consider 
suitable for assessing the cost and training effectiveness of 
a new, production-model simulator for initial skill- 
acquisition training. These methods may or may not be 
suitable for collecting the data needed to support the simu- 
lator design decisions that must be made in the early 
design phase of a simulator development effort. Similarly, 
the methods may or may not be suitable for assessing a 
simulator’s effectiveness for skill-sustainment training. 


Table 1 . Perspective and scope 


Perspective 

Behavioral sciences research 
Army aviation 
Focus 

New production model simulator evaluation 
Initial skill acquisition (basic/transition) 

Important topics not addressed 

Predicting training effectiveness from engineering data 
Utility of simulators for proficiency checking 
Utility of simulators for skill sustainment training 

Three important topics that I have not attempted to 
address except in passing include the feasibility of using 
engineering data to predict training effectiveness, the util- 
ity of simulator for proficiency checking, and the utility of 
simulators for skill-sustainment training. 

I will commence with a brief description of what I 
refer to as the “classic” transfer-of-training methods and 
an illustration of the types of data generated by them. 

Then, I will describe what I consider to be the key short- 
comings of these methods. Finally, I will describe a 
methodological approach that, in my view, is more effec- 
tive and efficient than the classic approach. 

It is important to emphasize that the approach I pro- 
pose does not eliminate the need to measure empirically 
the extent to which training in the simulator transfers to 
the parent aircraft. Rather, the approach is intended to 
insure that the simulator is functioning optimally and that 
the simulator training method is near optimal before an 
expensive transfer-of-training study is performed. Believe 
me, a researcher’s worst nightmare is to complete a 
transfer-of-training study costing hundreds of thousands 
of dollars, only to discover that the simulator was not 
functioning properly or that the trainees were given the 
wrong kind or amount of training in the simulator. 
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It is also important to emphasize that many of the 
methods and ideas I discuss are not new. If anything I 
have to say is truly a novel idea, it is the sequence in 
which the methods are used and the specific purposes for 
which they are used. 

Figure 1 illustrates the classic transfer-of-training 
research design. One group of trainees — the control 
group — receives no simulator training. The purpose of the 
control group is to provide information about the amount 
of time required to achieve proficiency through aircraft 
training alone. In addition to the control group, there are 
one or more groups of trainees who receive some amount 
of training in the simulator before being trained to profi- 
ciency in the aircraft; these groups are referred to as 
experimental groups. This illustration assumes that there 
are five experimental groups that differ only in the num- 
ber of hours of training they they receive in the simula- 
tor — 5 hours, 10 hours, and so on. AH groups are trained 
to the same level of proficiency in the aircraft, and the 
number of aircraft hours required to reach proficiency is 
recorded. 

A simulator is training-effective to the extent that 
simulator training reduces the amount of aircraft training 
required to achieve proficiency in the aircraft. In short, a 
simulator is training-effective to the extent that simulator 
training hours replace aircraft training hours. The hypo- 
thetical data presented in figure 2 illustrate the well- 
established relationship between the amount of simulator 
training the trainees receive and the amount of training 
required to achieve proficiency in the aircraft. The control 
group trainees, who receive no training in the simulator, 
require an average of 50 hours in the aircraft to reach 
proficiency; trainees who receive 5 hours of simulator 
training require only 40 hours in the aircraft to reach pro- 
ficiency. This negatively decelerating monotonic function 
illustrates the simple fact that each increment in simulator 
training time yields progressively less savings in aircraft 
training time. Data of this type are interesting, but are not 
sufficient to determine what amount of simulator training 
is optimal. 

Cost data must be brought to bear in deciding how 
much simulator training is enough. Figure 3 shows the 
relationship between the amount of simulator training and 
total training costs, or, its mirror image, cost savings. In 
producing this figure, I used the hypothetical training- 
effectiveness data shown in figure 3, along with the 
Army’s current estimates of the cost of an hour of Black- 
hawk simulator time and the cost of an hour of Blackhawk 
aircraft time. As you see, the simulator and aircraft costs 


are $338 and $1,424 an hour, respectively. The cost curve 
shows that very little cost reduction is realized from simu- 
lator training beyond 10 hours. If cost is the prime consid- 
eration, total cost can be minimized by giving each trainee 
15 hours of training in the simulator. However, if aircraft 
are unavailable for training, as many as 25 hours of simu- 
lator training can be given without increasing total train- 
ing cost appreciably. 

So, how can one find fault with a method that yields 
data like these? Let’s consider some of the problems. 

Table 2 lists some of the key shortcomings of the 
classic transfer-of-training method. First, the method 
yields only a composite measure of training transfer. This 
would not be a problem if the simulator were equally 
effective for training every maneuver. However, what is 
more likely is that training transfer for some maneuvers 
will be large and positive whereas training transfer for 
other maneuvers will be negligible or even negative. If 
this is indeed the case, the composite measure of training 
transfer is an underestimate of the simulator’s optimal 
training effectiveness. Stated differently, the cost effec- 
tiveness of the simulator could be increased by eliminat- 
ing training on those maneuvers for which training trans- 
fer is negligible or negative. 

Second, the relatively high cost of transfer-of-training 
studies prevents the use of this method for optimizing the 
various components of the training system. When the first 
version of a production simulator appears on the scene, 
there are going to be many uncertainties about how best to 
set it up and use it. For example: 

1 . Are all components of the simulator functioning 
as they were designed to function? 

2. Are there ways the simulator components can be 
adjusted or modified to increase the simulator’s training 
effectiveness? 

3. What maneuvers should be trained, in what order 
should the maneuvers be trained, and how much training 
should be given on each maneuver? 

4. What is the best method or procedure for training 
a given maneuver? 

5. What is the best way to employ the instructional 
support features available on the simulator? 

Although these questions are of critical importance, it 
would be prohibitively costly to answer them through 
classic transfer-of-training studies. Another more efficient 
method is required for this purpose. 

The third shortcoming is that transfer-of-training 
methods are not suitable for assessing some simulator 
training applications. Although a simulator may be highly 
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Figure 1. Classical TOT research design. 
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Figure 3. Total training costs/savings as function of simulator training hours (hypothetical data). 
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Table 2. Key shortcomings of classic transfer-of-training method 


1 . Yields only a composite measure of training transfer 

2. High cost prevents use for optimizing training system 

Simulator set up and functioning 
Type and sequence of maneuvers 
Training method and instructional support features 

3. Unsuitable for assessing some simulator training applications 


effective for training maneuvers that are too hazardous to 
perform in the aircraft, it is not possible to measure the 
extent to which such training transfers since it is not pos- 
sible to measure how well pilots can perform these haz- 
ardous maneuvers in the aircraft. For example, it is proba- 
bly too hazardous to measure in the aircraft a pilot’s abil- 
ity to recover from such emergencies as a brown-out or 
white-out, a dual engine failure, a complete loss of tail- 
rotor effectiveness, or a severe wind sheer. 

There are other maneuvers and conditions for which 
proficiency measurement in the aircraft is excessively 
costly, even if the risk is acceptable. For instance, measur- 
ing pilots’ ability to perform takeoffs and landings at high 
surface elevation may be costly if the research is not con- 
ducted at a location that is close to mountainous terrain. 
Also, because visibility conditions in the real world can- 
not be controlled, it may be excessively costly to measure 
pilots’ ability to perform maneuvers under specific 
degraded visibility conditions. 

The flow diagram shown in figure 4 illustrates my 
views about the type, sequence, and purpose of research 
studies that, together, may eliminate some of the short- 
comings of the classic transfer-of-training methods. This 
approach to simulator evaluation is the result of a large 
amount of thought and a small amount of data collection, 
so it is not presented here as a proven research method. 
Although my colleagues and I believe the approach is 
workable and sensible, I invite all of you to critique the 
approach and to let me know what doesn’t make sense to 
you. 

The four small shadowed boxes in figure 4 identify 
four types of research studies that I consider necessary for 
the efficient assessment of a simulator’s training and cost 
effectiveness; the boxes with the rounded corners identify 
the purpose served by each of the four types of studies. 

As you can see in the upper left corner, the purpose 
of the analytical studies is to identify maneuvers for 
which training transfer cannot be assessed either because 
the maneuver clearly cannot be trained in the simulator, or 
a pilot’s proficiency on the maneuver cannot be measured 
in the aircraft without unacceptable risk or cost. For obvi- 


ous reasons, these maneuvers must be excluded from a 
transfer-of-training study. The purpose of the next two 
types of studies is to insure that the simulator and the 
simulator training are near optimal before a transfer-of- 
training study is commenced. Because of the limited 
amount of time available, I will not comment further on 
the analytical studies. Instead, I will use the time I have 
left to discuss the rationale and procedures for the three 
remaining studies: backward transfer, in-simulator skill 
acquisition, and modified transfer of training. 

The idea behind a backward-transfer study is a simple 
one (table 3). If forward transfer is the extent to which 
training in a simulator transfers to the parent aircraft, 
backward transfer must be the extent to which training in 
the parent aircraft transfers to the simulator. If the skills 
required to perform a maneuver in the parent aircraft are 
the same as the skills required to perform that maneuver 
in the simulator, one would expect a high degree of back- 
ward transfer. If backward transfer is not high, it is rea- 
sonable to assume that something about the simulator is 
not right. In short, the fundamental premise is that a low 
backward transfer indicates one or more important short- 
comings in the simulator. About 30 years ago. Jack 
Adams and his colleagues at the University of Illinois 
considered the feasibility of using measures of backward 
transfer to predict the degree of forward transfer. 

Although backward transfer may indeed be a reasonably 
valid predictor of forward transfer, it is important to 
emphasize that predicting forward transfer is not the pur- 
pose for which backward-transfer studies are proposed 
here. 

The procedure for conducting a backward transfer-of- 
training study is simple and straightforward. The first step 
is to select pilots who are highly experienced in the parent 
aircraft and who have had little or no experience in simu- 
lators, especially in the simulator being evaluated. The 
next step is to evaluate each pilot’s proficiency in the air- 
craft for each maneuver to be evaluated in the simulator. 
The third step is to measure the pilots’ initial proficiency 
on each maneuver in the simulator. Initial proficiency 
refers to how well the pilots perform on no more than the 
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Table 3. Backward-transfer studies 


Concept 

Measure aircraft-to-simulator transfer (experienced aviators) 

Premise 

Low backward transfer indicates simulator shortcomings 
Procedure 

Select pilots with long aircraft experience and no simulator experience 
Assess task proficiency in aircraft (desirable) 

Measure initial task proficiency in simulator (one to three iterations) 
Assess backward transfer 
Interview pilots 
Benefits 

Efficient (time and cost) 

Yields diagnostic data about simulator shortcomings 


first three attempts. There is a substantial amount of evi- 
dence that indicates that experienced pilots are able to 
adapt very quickly to even substantial differences between 
the aircraft and the simulator; as a result, a pilot’s perfor- 
mance may quickly become contaminated by simulator- 
specific learning. The fourth step is to assess the degree of 
backward transfer by comparing simulator performance 
with aircraft performance, published performance stan- 
dards, or both. The final step is to question pilots about 
the reasons for any poor performance in the simulator. 

If the results reveal simulator shortcomings that can 
be eliminated completely or in part, the simulator can be 
modified and backward transfer can be measured again 
for the maneuvers that were performed poorly. 

Backward-transfer studies have two important bene- 
fits. First, they are highly efficient in terms of both cost 
and time. If necessary, further cost reductions can be real- 
ized by eliminating proficiency measurement in the air- 
craft. The results of our backward-training research indi- 
cate that proficiency measurement in the aircraft is useful 
but not essential. Second, backward-transfer studies yield 
data that are useful in determining the reasons for poor 
simulator performance. In addition to the judgments of the 
participating pilots, much can be learned about simulator 
shortcomings by studying the types of errors made in 
performing a maneuver and the manner in which simula- 
tor performance differs from aircraft performance. 

Figure 5 presents an example of the kind of results 
that can be expected from a backward-transfer study. The 
study was the first step in evaluating the effectiveness of 
the AH-1 Flight and Weapons Simulator for sustaining 
proficiency on emergency touchdown procedures. The 
15 pilots who participated in the study were highly expe- 


rienced AH-1 instructor pilots. The solid bars show the 
mean ratings for performance in the aircraft; the cross- 
hatched bars show the mean ratings for the first attempt to 
perform the same maneuvers in the AH-1 simulator. A 
rating of 1 indicates clearly unacceptable performance — a 
crash, a hard landing, landing short, and so on. A rating of 
7 indicated the level of performance that the evaluators 
expected of the average AH-1 instructor pilot. 

The ratings of aircraft performance indicated that the 
various emergency touchdown procedures differ in their 
inherent difficulty — the simulated anti-torque failure 
appears to be the most difficult maneuver, and the shallow 
approach to a running landing appears to be the least dif- 
ficult maneuver. You can see that the ratings of simulator 
performance are far lower than the ratings of aircraft per- 
formance. More important, there is little correlation 
between the simulation ratings and the aircraft ratings. For 
instance, although most aviators performed standard 
autorotations very proficiently in the aircraft, no aviator 
received a rating higher than 1 on a standard autorotation 
in the simulator. 

Although these results are not definitive proof that 
the AH- 1 simulator is ineffective for training emergency 
touchdown procedure, they leave no doubt that the simu- 
lator and the aircraft differ in ways that may have a major 
influence on training effectiveness. In truth, it is not possi- 
ble to examine these findings without worrying about 
negative transfer. 

Table 4 shows a tally of the IP’s spontaneous com- 
ments about the factors that contributed to the poor per- 
formance in the simulator. It can be seen that most of the 
IPs attributed their poor performance, in part, to the lack 
of visual cues needed to operate near the ground. The 
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Table 4. Factors contributing to low backward transfer 3 


Lack of visual cues 

Visual display blurred near ground (100%) 

Unable to judge altitude near ground (94%) 
Insufficient visual cues to maintain hover (87%) 
Entry point difficult to judge (81%) 

Lack of peripheral cues (69%) 

Unrealistic response to control inputs 
Response to collective inputs (75%) 

Response to cyclic inputs (63%) 

a N = 15 for all percentages. 

study was conducted in one of the early AH-1 simulators 
that was equipped with a camera-model -board visual sys- 
tem. The comments of the IPs are consistent with the 
results of tests that have shown that the camera-model - 
board system has poor focus and resolution when the 
probe is located very close to the model board. Table 4 
also shows that most of the IPs identified unrealistic 
response to collective and cyclic inputs as an important 
contributor to poor performance in the simulator. 

Although pilot judgments have not always proved to 
be highly reliable sources of information about simulator 
functioning, it would be foolish to ignore judgments that 
are as consistent as the ones shown here. 

As I define the term, an in-simulator skill-acquisition 
study is a study performed to determine ( 1 ) how much 


simulator practice is required to gain proficiency on a 
given maneuver, and (2) the maximum level of profi- 
ciency that can be achieved (table 5). The recommenda- 
tion to conduct skill-acquisition studies is based on two 
premises. The first premise is that the cost effectiveness of 
a simulator can be degraded significantly by inefficient 
simulator training. Inefficient simulator training may be 
the result of such factors as (1) too much or too little 
simulator training, (2) the use of inefficient training meth- 
ods, and (3) the expenditure of an excessive amount of 
time on training maneuvers for which skill acquisition is 
very slow. The second premise is that skill acquisition 
data can be used to optimize simulator training. 

Before proceeding, I would like to comment briefly 
on a couple of issues. The first is the importance of 
determining the optimal amount of simulator training for 
each maneuver. It is obvious that money is wasted when 
training on a maneuver is continued beyond the point at 
which performance asymptotes. What is not so obvious is 
that overtraining on a maneuver may actually reduce 
training transfer. Jack Dohme, an Army Research Institute 
researcher at Fort Rucker, has shown me unpublished data 
that strongly suggest that too much simulator training on a 
maneuver can, in fact, reduce training transfer. 

On the other hand, there are reasons to believe that 
too little simulator training on a task may create problems 
of a different kind. The problems stem from the fact that 
some minimum level of proficiency on some maneuvers is 


Table 5. In-simulator skill-acquisition studies 


Premises 

Cost effectiveness of simulator degrades by training inefficiencies 
Too much/little simulator training 
Ineffective training methods 

Time spent training maneuvers for which skill acquisition is slow 
Simulator training can be optimized using skill acquisition data 
Procedure 

Select trainees (novice and experienced aviators) 

Measure practice-iterations/time-to-criterion as function of maneuver type/sequence, training procedures 
Benefits 

Yields data with which to specify near-optimal training 
Maneuver sequence 
Practice iterations 
Training procedures 
Efficient (time and cost) 

Identifies maneuvers that should be excluded from simulator training 
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required to learn other, more complex maneuvers effi- 
ciently. For example, instructor pilots claim that efficient 
learning of out-of-ground-effect hover is not possible until 
a student is reasonably proficient at performing 
in -ground-effect hover. 

The second issue is the importance of establishing 
optional training methods. Many persons believe that all 
simulator training should be conducted in the context of a 
training scenario that approximates an aircraft training 
flight. Training in the context of a scenario of this type 
invariably wastes a lot of time in traveling from one point 
to another. For instance, training on approaches and land- 
ings in a simulator need not require the trainee to fly the 
entire traffic pattern in order to get the needed practice on 
the final approach and landing. Using the simulator’s 
“initial condition set” to place the simulated aircraft on 
the final approach leg can greatly increase the number of 
practice iterations that can be accomplished during a train- 
ing period. Although training method is certain to have a 
major effect on training efficiency, few studies have been 
conducted to assess the relationship between training 
method and rate and level of skill acquisition in the 
simulator. 

Now let us discuss the procedures for conducting 
skill acquisition studies (refer to table 5). The procedures 
are simple. The first step is to select the pilots who are to 
participate in the study. Normally, the study would be 
conducted only with novice aviators who have no experi- 
ence in the simulator. However, we have found it useful 
also to investigate the skill acquisition of pilots who are 
highly experienced in both the simulator and the parent 
aircraft. The use of experienced aviators is an efficient 
way to determine the maximum level of proficiency that 
is possible for a given maneuver. 

The second step is to measure the number of practice 
iterations and the amount of training time required to 
reach a prescribed level of performance on each maneu- 
ver. Since the purpose of the skill-acquisition study is to 
optimize training methods, the practice iterations and 
training time would be measured as functions of such 
independent variables as type of maneuvers, the sequence 
in which maneuvers are trained, and the training proce- 
dures used. 

Skill-acquisition studies have three kinds of benefits. 
As I have already mentioned, the main benefit is that the 
data can be used to specify a near-optimal training method 
before a transfer-of-training study is commenced. The 
second is that skill-acquisition studies are very efficient 
relative to transfer-of-training studies. A third benefit is 


that the data can be used to identify maneuvers that should 
be excluded from simulator training because skill acquisi- 
tion in the simulator is slow or nonexistent. 

I would like to take a few minutes to show you the 
results of a skill-acquisition study we performed on the 
AH-1 Flight and Weapons Simulator (fig. 6). The ultimate 
objective of the study was to assess the utility of the simu- 
lator for sustainment training, so we measured the simula- 
tor skill acquisition of experienced AH-1 pilots rather than 
trainees. Because we had not conducted skill-acquisition 
studies before, we assumed that experienced pilots would 
require no more than 10 practice iterations to reach profi- 
ciency on any task. So, the entire schedule was set up to 
obtain data on only 10 iterations. This assumption turned 
out to be grossly incorrect. In fact, more than 10 iterations 
were required to reach proficiency on most maneuvers. As 
a consequence, it was necessary to use regression analysis 
to project the number of practice iterations required to 
reach proficiency. Figure 6 shows projected iterations to 
proficiency for each of 15 maneuvers. For three maneu- 
vers, there was no measurable learning during the first 
10 iterations, so no projections could be made for the 
maneuvers. For the remaining maneuvers, the projected 
numbers of iterations to proficiency varied from 9 to 27. 

Results such as these are useful for making decisions 
about the kinds of maneuvers that should be trained in the 
simulator and the amount of simulator time required to 
accomplish training on each maneuver. In addition, such 
results lead to some interesting questions about the design 
and function of the simulator. For instance, why do skilled 
aviators require so many trials to master normal 
approaches and hover tasks in the simulator? 

The final and most critical study in the sequence is a 
transfer-of-training study. Table 6 shows my views about 
ways in which the classic transfer-of-training method can 
be modified to produce more useful data. Some involve 
changes in the simulator training and some require 
changes in the aircraft training. 

There are three ways in which simulator training 
should be changed. First, I believe that all trainees should 
be trained to a prescribed level of proficiency in the simu- 
lator rather than receive some pre-defined amount of sim- 
ulator training. Second, the amount of simulator training 
should be varied by varying the number of maneuvers 
trained rather than spreading fewer and fewer hours of 
training over some fixed number of maneuvers. And third, 
I believe that good estimates of cost effectiveness are 
possible only if the researcher is careful to record the 
nonproductive training time spent in the simulator. The 
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Table 6. Modified transfer-of-training study 
(key differences from classic TOT study) 


Simulator training 

Train to proficiency on each maneuver 
Record nonproductive training time 
Crash re-set 
Repair 

Procrastination, etc. 

Aircraft training 

Record iterations to proficiency for each maneuver 
Record nonproductive training time 
Transit and refueling 
Performing maneuvers already mastered 
Procrastination, etc. 


apparent cost of simulator training can be increased sub- 
stantially by such extraneous events as crashes, simulator 
failures, and procrastination by instructors or students. 

Next, consider the aircraft training procedure. I think 
it is essential to monitor and record iterations-to- 
proficiency on each maneuver trained in the aircraft. Dur- 
ing aircraft training, a trainee simply cannot avoid per- 
forming certain maneuvers even though they already have 
been mastered. For instance, a trainee cannot accomplish 
a training flight without performing at least one takeoff 
and one landing. Hence, the total number of maneuver 
iterations performed during aircraft training is not deter- 
mined by a trainee’s training needs alone. In short, the 
effect of simulator training on the amount of aircraft train- 
ing required cannot be determined without knowing the 
point at which the trainee reached proficiency on each 
maneuver. 

As was true for simulator training, I believe it is nec- 
essary to record nonproductive training time for aircraft 
training. The quality of the aircraft cost data can be 
improved by subtracting from total aircraft hours the 
amount of time spent traveling between training sites, the 
time spent refueling, the time spent performing maneuvers 
already mastered, the time wasted because of procrastina- 
tion, and so on. 

A transfer-of-training study with the changes recom- 
mended here should provide the data needed to determine 
transfer-of-training by maneuver and by blocks of maneu- 
vers. Moreover, the cost effectiveness of a simulator can 
be computed as a function of the specific maneuvers 
trained in the simulator. Finally, the cost-effectiveness 


estimates will not be confounded by unproductive time 
spent in the simulator, or in the aircraft, or both. 

That concludes my remarks about training effective- 
ness assessment. Before inviting questions I would like to 
thank the sponsors of the workshop for giving me an 
opportunity to test my views before such a large body of 
experts. And, I would like to thank those of you in the 
audience for your kind attention. 

MR. McGOWAN: On these backward-transfer-of- 
training studies, how do you account for a situation in 
which a maneuver, let’s say AFCS-off flight in a heli- 
copter simulator, may actually be easier in the simulator 
than it is in the aircraft, and how would you catch that in 
such a study? Does that question make sense? 

DR. CROSS: Yes, Greg, your question certainly does 
make sense. And you have pointed out one shortcoming 
of backward-transfer studies. The results of a backward 
transfer study enable you make a one-sided decision. If 
you have a high degree of positive transfer you cannot 
conclude that everything is right with the simulator. It is 
possible that a task is so easy to perform in the simulator 
that it doesn’t even come close to representing its cor- 
responding task in the aircraft. In the example you gave, I 
don’t know exactly why AFCS-off flight in a simulator is 
easier. I don’t remember that our results show that to be 
the case. 

MR. McGOWAN: No, I am not saying that is the 
case. I am just saying that could be the situation. 

DR. CROSS: Oh, I see. My answer is still relevant. If 
you have a task that is unrealistically easy to perform in 
the simulator, it is unlikely that the simulator would pro- 
vide effective training on that task. Unfortunately, 
backward-transfer studies are not effective in identifying 
that kind of problem. Such a problem might be revealed 
by in-simulator skill-acquisition studies, and most cer- 
tainly would be revealed by transfer-of-training studies. 

MR. HART: You used the Huey simulator, which 
apparently only poorly duplicates the helicopter. If you 
did the same study, let’s say with a more modem simula- 
tor, would you get similar results? It seems to me that the 
problem in backward transfer has to do with the lack of 
authenticity of the simulator itself. Is that accurate? 

DR. CROSS: No, it is not. I may have said Huey; if I 
did, I apologize. The backward-transfer and the in- 
simulator skill-acquisition studies were conducted in the 
AH-1 flight simulator, which is far more sophisticated 
than the old Huey simulator. 
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MR. HART: But again, wouldn’t the results vary sig- DR. CROSS: It is perfectly accurate. That is the fun- 

nificantly as the quality of authenticity improves? Is that damental premise underlying all these kinds of studies 
not an accurate statement? that I have discussed today. 



Kenneth Cross has been engaged in human factors research for 27 years. He has been 
at Anacapa Sciences since 1970 and now serves as Anacapa’s president. Dr. Cross 
has formal training in research psychology and statistics; he received his doctorate 
degree from Kansas State University. Before joining Anacapa Sciences, he was 
research coordinator at the Naval Missile Center’s Human Factors Laboratory. His 
research has dealt mainly with human performance in complex military systems. 
Much of Dr. Cross’s time over the last ten years has been spent conducting studies of 
Army helicopter training at the U.S. Army Aviation Center, with emphasis on assess- 
ing the effectiveness of training conducted in helicopter simulators. 
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The ability of rotorcraft pilots to hover and maneuver 
with agility in slow speed flight has placed unique and 
complex requirements on simulator manufacturers to 
demonstrate the authenticity of their product for the pur- 
pose of gaining rotorcraft training credit. 

The FAA’s evaluation of a simulator’s capability is 
further complicated by the fact that the FAA does not 
have the resources to collect and compare the static and 
dynamic flying-qualities data that are required to conduct 
a comprehensive analysis. As a result, the FAA resorts to 
the practical approach of assigning qualified pilots to fly a 
flight simulator for the purpose of determining its value as 
a training device. Restated, pilots and engineers operate 
and otherwise evaluate flight simulators and render opin- 
ions about the adequacy of the simulator in terms of its 
proposed use and the credits requested. There are many 
other important objective measures of adequacy, but the 
importance of the subjective evaluation conducted by the 
pilot cannot be overstated. 

This subjective portion of the evaluation may be 
enhanced by following the procedures suggested below. 
The details of a method for collecting and graphically cor- 
relating subjective ratings will be presented. The process 
has been tailored to aid engineers in their efforts to define 
the training value and limits of a given simulator with a 
substantially improved degree of confidence. 

The FAA pilot’s job is to define the simulator. Ide- 
ally, the pilot should be able to characterize the simulator 
in a format that can be understood by engineers and regu- 
lators. The evaluation pilot’s insight into the real aircraft 
and its operational applications can be useful in helping 
engineers establish an appropriate scope of test to insure 
that the important flight phases and environmental condi- 
tions are considered. 

The evaluation of rotorcraft flight simulator devices 
during up-and-away operations is seldom critical to the 
determination of overall suitability. This is because the 
aircraft is generally stable, and the quality of the visual 


scene is often not critical to the learning experience. In 
contrast, the slow-speed regime is critical because most 
helicopter-unique training experiences occur in the slow- 
speed regime. In addition, the helicopter is least stable at 
these speeds, and the visual-motion system cues are most 
difficult to reproduce. 

Relaxed slow-speed maneuvering high above the 
ground decreases the demand on the visual scene. In con- 
trast, precision hover operations, low over a textured sur- 
face, place the greatest demand on the simulator’s visual 
scene and motion system. In short, the evaluation pilot 
must investigate the authenticity of the simulator during a 
variety of maneuvers, including precision hover and 
during aggressive maneuvers, such as quick stops and 
inadvertent, uncommanded heading reversals (weather- 
cocking into a tailwind). 

Although simulators are also very useful for teaching 
emergency procedures (such as tail-rotor failure), the val- 
idation of these events in a simulator dictates the use of 
quantitative data to determine reasonableness. A quantita- 
tive analysis is the only practical validation technique for 
such an event since there is normally little opportunity for 
pilots to build up an adequate (failure-mode) experience 
base in a real aircraft for use in an evaluation of the char- 
acteristics designed into a simulator. 

The pilot assessment of suitability has historically 
been a key factor during the evaluation of aircraft by the 
FAA. The importance of this activity is difficult to over- 
state. Thus, before proceeding, it is useful to take a brief 
look at current procedures to establish a common point of 
departure. 

Although research pilots and military test pilots tend 
to employ pilot rating scales, FAA pilots typically do not. 
The FAA pilot’s task is to determine if the aircraft and its 
systems are safe. They make determinations about the 
adequacy or suitability of an aircraft for civil operations. 
There really is little call for pilot rating data per se. In 
addition, FAA pilots are primarily interested in workload. 
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and the basic pilot rating scale is not well suited to such 
an application. Finally, when the pilot ratings of several 
pilots are compared, they often do not agree, and such 
disagreements tend to bring the validity of the entire eval- 
uation into question. 

In short, the lack of a usable (FAA-oriented) pilot rat- 
ing scale and the historical problems stemming from 
scatter in the data have produced deterrents to the general 
use of pilot ratings. These deterrents need to be eliminated 
before FAA pilots and engineers can be expected to 
embrace an evaluation method for flight simulators that 
involves pilot ratings. 

There are many explanations for disagreements in 
pilot subjective ratings, and though some scatter in the 
data is normal, all evaluations should be conducted so as 
to minimize the scatter in the ratings. This presentation 
deals at great length with this issue and offers techniques 
to minimize scatter in the data when a number of pilots 
are employed on the same evaluation. 

The method presented is based on the premise that if 
an engineer asks two equally qualified pilots the very 
same question, the result will be a common answer (pilot 
rating). A sloppy approach to staging a rating question to 
a number of pilots will in turn produce scatter in the 
results. That is, the proposed method introduces a disci- 
pline to the evaluation process. 

Nevertheless, all scatter cannot be eliminated, nor 
should it be. Some apparent scatter in the data is not scat- 
ter at all, it is more data. For example, some disagreement 
in ratings may be explained by examining the background 
of the pilots. One pilot may be much more qualified in the 
aircraft than the others. Alternatively, one pilot may have 
used a different piloting technique and effectively 
changed the task. There is almost always a reason for 
apparent scatter that is not eliminated by the discipline to 
be proposed. 

Pilots evaluate simulators by manipulating them as 
though they were flying a real aircraft in the conduct of a 
real mission task. Some operations are conducted single- 
pilot, some are two-pilot operations. Some flights are 
conducted with all systems operative, others are con- 
ducted with a variety of failures. Some tasks are very 
relaxed. Some relaxed flight tasks are made more difficult 
by the need to accomplish a number of secondary tasks at 
the same time. Other tasks require a great deal of preci- 
sion interaction with the vehicle. Regardless of the basic 
circumstances, if the evaluation pilot is not required to 
work hard, there will be little potential for the kind of 
stress required to obtain a useful evaluation. 


For example, a relaxed task such as a cross country 
flight, 1 ,000 feet above rolling terrain, bathed in bright 
sunlight, may not introduce sufficient workload to detect 
the shortcomings of a given simulator. Gusty winds will 
increase the workload. Decreasing visibility will also 
increase the workload. The introduction of factors that 
produce increasing levels of workload result in stress and 
enable pilots to find faults which allow them to become 
more discriminating in their assessments of a simulator’s 
performance and related authenticity. 

The fact is, pilots train to insure that they are able to 
cope with adversity in flight. They learn how to fly 
instrument approaches, and how to provide compensatory 
control inputs to suppress the gust response of their air- 
craft in the real world. Pilots must learn how to fly and 
deal with failure modes in a variety of environments. 
Anyone can quickly learn to fly almost any kind of air- 
craft on a clear day under calm conditions. Darkness, tur- 
bulence, and aircraft failure modes stress the pilot’s ability 
to maintain safe flight conditions. It seems reasonable that 
one of the objectives of simulation should be to provide a 
pilot with the opportunity to experience a variety of 
adverse (stressful) combinations of flight environments 
and failure modes with the intended purpose of accelerat- 
ing the learning process, aging the pilot to maturity in the 
least calendar time and at a minimum expense to the 
employer, and at the same time maintaining maximum 
safety by minimizing accident exposure in actual flight 
during abnormal and emergency operations. 

Figures 1(a) and 1(b) illustrate the variety of unique 
conditions which collectively define the environment 
within which a pilot can be expected to fly a rotorcraft. 
These environmental conditions can be used in a variety 
of visual conditions. The authentic duplication of these 
environments may dictate that a simulation device have a 
large repertoire of visual scenes. After some analysis, one 
might conclude that the availability of a large number of 
discrete visual scenes is not as important as the authentic- 
ity of the scenes available in the simulator. Repeatability 
of specific scenes in the simulator is also useful when 
analyzing the effect of variables such as pilot experience 
and training levels on the ability of crews to accomplish 
specific maneuvers. Waiting with a real aircraft for spe- 
cific meteorological conditions (in the real world) to be 
repeated to derive similar data can be prohibitively 
expensive. 

A moonless, starless flight over a dark sea is easy to 
simulate. The world is dark. Daylight scenes are more 
difficult. Images of trees, buildings, and runways as 
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Figure 1. Characteristics defining operational environment. 
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observed through a haze may or may not be authentic; it is 
difficult to know. Maybe we don’t even care if such 
scenes are authentic. The need for a sharp representation 
of microtexture during a low hover, on a bright day, is 
often very difficult to authentically simulate. This may be 
one of the most significant conditions to evaluate, for a 
failure to achieve the desired authenticity in the low- 
altitude, daylight environment may preclude the accom- 
plishment of a precision hover training task. 

The introduction of turbulence into this task (envi- 
ronment) can prevent a pilot from accomplishing a preci- 
sion hover task in some real helicopters. Thus, the intro- 
duction of turbulence reduces the expectations of the pilot 
where he no longer expects to do well in the simulator 
either. Here the introduction of turbulence into a simula- 
tion event has the potential of masking some simulator 
problems because of decreased expectations. The point: 
one must be careful in the use of environmental variables. 
We will return to the environment later. 

Systematic reports of subjective evaluations typically 
employ pilot rating scales. The most popular pilot rating 
scale is referred to as the Cooper-Harper pilot rating scale 
(see fig. 2). With ratings ranging from 1 to 10, it is the 
basic scale for most aircraft flying-qualities research work 
accomplished today. This an excellent scale, supported by 
40 or more years of experience, but it lacks the detailed 
definition required for the evaluation of simulation 
devices. The range of this scale extends beyond the scope 
(or typical needs) of most FAA evaluations of simulation 
devices. 

It is conceivable that the pilot of a certified civil heli- 
copter may experience a situation to which a rating of 7 
could be assigned, but even 7s should be rare. A rating of 
7 means that the pilot was in control, but that the pilot was 
working as hard as possible, and that the resulting perfor- 
mance was inadequate. 

At the other extreme of the scale, the pilot rating of 1 
is reserved for highly automated flight-control systems or 
extremely relaxed tasks. In summary, pilots actively con- 
trolling certificated aircraft (with no system failures) in 
normal operational environments are expected to assign 
ratings that range between 2 and 5.5. Pilots evaluating 
automated flight-path control may assign 1 and 1.5. Seri- 
ous flight-control failures, or very adverse operating envi- 
ronments, or difficult combinations of failure mode and 
bad environments, may produce pilot ratings of 6 or more. 

Figure 3 shows a scale that has been expanded to 
meet the needs of the FAA for the evaluation of civil 
rotorcraft operations. This rating scale is only a sugges- 


tion; it has not been endorsed by the FAA and there is 
every reason to expect that it can and should be improved. 
Nevertheless, the added detail is intended to help a group 
of pilots produce more consistent results by minimizing 
the opportunity for scatter in the data caused by individual 
interpretation of the Cooper-Harper scale. 

When you compare the scale in figure 2 with the 
scale in figure 3, be advised that they are the same scale. 
The words in figure 3 are meant to expand upon the words 
in figure 2. They are intended to provide pilots with a bet- 
ter understanding of the meaning of the very brief state- 
ments in figure 2. Also note that the expanded scale pro- 
vides definitions for ratings of 1 .5, 2.5, 3.5, etc., whereas 
figure 2 does not. These additional half-ratings are not the 
invention of the author; they have been used from the 
beginning of time. The use of half-ratings is required, 
because most ratings range between 2 and 5. Experience 
has shown that the rating scale has been used as a kind of 
shorthand for pilots to communicate with engineers and 
other pilots. It is used to report the results of research that 
involves many, many variations in the evaluation task or 
characteristics of the aircraft. The half-numbers increase 
the number of “quality steps” available within a given 
small range of ratings to allow pilots to achieve the 
desired discrimination or hierarchic ranking of evaluation 
situations. These additional quality steps also allow the 
pilot to more accurately report the effect of variations in 
the environment on pilot-aircraft performance. 

Pilots should not be required to commit the scale to 
memory, but pilots should make an effort to develop an 
awareness of the scale. They then should be allowed to 
look at the scale during the debriefing period following a 
flight evaluation. At that time, the pilot should rate the 
simulator experiences. This process will be developed in 
detail later. 

Assume that a team of four pilots has been selected to 
evaluate a simulator. Their first step is to refresh their 
knowledge of the aircraft. If they are very familiar and 
current in that respect, this step is accomplished from 
memory. But for this example, assume that all of these 
pilots need to fly the aircraft. The first pilot, Green, con- 
ducts the hover-landing task described on the “Pilot Data 
Card” under the four conditions identified in figure 4 as 
A, B, C, and D. 

Each time a pilot conducts the task, the factors that 
define the situation are recorded. Next, an assessment is 
entered for each situation. In this example, the assess- 
ments have ranged from a rating of 2 for a “clear day, 
calm air” to a 6 for an “overcast nighttime” situation. The 
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subphases with accompanying conditions. 
Figure 2. Cooper-Harper pilot rating scale. 
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From time to time, the pilot may instruct the autopilot. System achieves long 
and short term objective with no pilot input directly to the conventional flight 
controls; inputs are selected via secondary (electronic) controls. The quality 
of flight path performance is self-monitored and alerts are provided to the pilot 
when he needs to take over; first and second failures are fail operate. Auto- 
matic mode shifting is provided (i.e., cruise to glideslope or glideslope to go 
around). 


From time to time, the pilot may instruct the autopilot. System achieves long 
and short term objective with no pilot input directly to the conventional flight 
controls; inputs are selected via secondary (electronic) controls. The quality of 
flight path performance is self-monitored and alerts are provided to the pilot 
when he needs to take over; first failure is fail operate; second or third failure 
one fail passive. Pilot is required to make occasional long term trim adjust- 
ments in one or two controls during transitional flight or during mode shifts. 


System achieves long term and short term gust suppression objectives with 
little or no pilot input directly to the conventional flight controls; inputs are 
often accomplished via secondary (electronic) controls. The quality of flight 
path performance is self-monitored and alerts are provided to the pilot when 
he needs to take over. Monitoring of short and long term response con- 
tinous but relaxed. Pilot may be required to occasionally adjust one axis/para- 
meter during the performance of precision maneuvers or during major flight 
path changes. 


The pilot is continually involved in monitoring the short and long term perfor- 
mance of the aircraft. Deviations develop slowly and in a predictable way, and 
can be eliminated quickly with relaxed control techniques. Errors generally 
develop along or about one axis at a time. 


The pilot is continually involved in the short-term control of the aircraft. Two or 
more controls are typically displaced in a sequential pattern. The aircraft can 
be trimmed with no more than one parameter/control needing attention at any 
given time. Control techniques are relaxed and pilot compensation is predict- 
able and easy but requires continuous involvement. 


There is a characteristic that occasionally requires heightened attention, 
potentially disrupting the pilot's scan or control technique and momentarily 
taking precedent over other tasks. The aircraft is just a bit less predictable, 
possible because of problems trimming or due to an inconsistent response to 
gusting winds. 


Moderate pilot compensation is required. For relaxed flight phases, the 
control activity required is clearly achievable, but the effort produces im- 
patience with the task and fatigue. Adjusting one control may require adjust- 
ments in other controls. For precision tasks, the workload contributes to 
occasional errors and excessive deviation. 


Moderate pilot compensation is required to achieve desired performance. 
There are one or more clearly annoying characteristics that make relaxed 
control clearly unachievable. On occasion, the desired performance is not 
achieved without considerable pilot compensation. 



97 






















Simulator Data Transfer 



Moderately 

Objectionable 

Characteristics 




Very 

Objectionable 
But Tolerable 
Characteristics 


6.5 



8.0 


9.0 


Unacceptable 

Control 

Characteristics 


10.0 


Considerable pilot compensation is required to achieve adequate perfor- 
mance. For cruise, the control activity required is clearly achievable, but failure 
to stay attentive may result in the need to recover from an unusual flight con- 
dition. In precision tasks, the pilot is not pleased with aircraft performance and, 
if given the option, would probably fly slower/faster, etc., to improve perfor- 
mance. A pilot would not routinely plan to depart on a flight involving this level 
of effort. 

Adequate performance requires almost total involvement in the flight-control 
task. Failure to stay attentive will probably result in an unusual attitude. The 
pilot is confident about performing single flights under this workload, but 
would not routinely plan to fly an aircraft requiring this workload. If encountered 
unexpectedly, the pilot would not expect to fly at this level of effort for more 
than 15 minutes during precision tasks or 120 minutes during non-precision 

tasks. 

Extensive pilot compensation is required: The pilot is totally involved in 
control task, scan rate is at its limit, and pilot is moving two or more controls 
continuously. The pilot is alarmed and expects to experience periods where 
performance represents marginally safe flight. Pilot would not willingly fly at 
this level of effort for more than 1 0 minutes for precision tasks or 60 minutes 
_dunn2jTonjgrecision > tasks i<iiii _^ — 

Extensive pilot compensation may not yield adequate performance. Work- 
load is so high and performance is so marginal that the pilot would not con- 
tinue to pursue the task unless there were no other alternatives. In the landing 
task, the aircraft will probably experience minor damage, without crew or 
passenger injury. 

Adequate performance is not attainable with maximum tolerable pilot compen- 
sation. Gross control of the aircraft is not in question, however, if the pilot 
persists at this level of workload, the safety of the aircraft is clearly in question. 

In the landing task, the aircraft will receive damage and there may be personal 
injury. 

Maximum achievable pilot compensation will not produce adequate perfor- 
mance; even lor brief periods. Gross control of the aircraft is sometimes a 
concern. If the pilot persists, performance will deteriorate due to fatigue, and 
the aircraft may receive serious damaged. Personnel are at serious risk. 

Adequate performance is clearly unachievable with maximum pilot compen- 
sation, even for short periods of time. Considerable pilot compensation is 
required to retain control and transition to a less demanding task. The ability 
to transition out may be in question. Crew is at risk but will probably survive. 

Adequate performance is clearly unachievable. If the pilot persists, gross 
control of the aircraft will probably be lost for brief periods and then regained. 
Maximum achievable pilot compensation may not be adequate to transition to 
a less demanding mode of flight. Crew and passengers will probably survive 

with injury, even if the aircraft is lost. 

If the task is attempted, control will be lost and probably never regained in 
time to return to normal flight. Such events typically result in a catastrophic 
loss of the aircraft. 


Figure 3. Concluded. 
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TASK SHORT TITLE 

PLATFORM HOVER-LANDING 

PILOT DATA CARD SIM FLT □ 

Pilot Name: GREEN A/C FLT [Z] 

TASK: Low hover in confined area. Landing on a platform one hundred feet above a water surface. 

Obstructions are present ahead and to the right. Upon landing rotor clearance is 30 feet to 
closest obstruction. Steel structure rises ahead. 

SITUATION 
ID CODE 

FACTORS DEFINING THE 
TASK ENVIRONMENT SITUATION 

PILOT 

ASSESSMENT 

(RATING) 

A 

Clear Day, Calm Air. 

2 

B 

Clear Day, 1 0 KT RT Cross Wind. 

2.5 

C 

Clear Day, 10 KT RT Cross Wind, Gusting to 17 KT. 

5 

D 

Night, Overcast, no surface lights, single landing LT, 
1 0 KT RT Cross Wind, Gusting to 1 7 KT, (see Note 1 ) 

6 

OPERATING STATE: Normal 

CONFIGURATION: Midwt, mid C.G., Doors closed 

Note 1: Tower obstruction lights, landing pad edge lights. 


Figure 4. Pilot rating data card for assessing one aircraft for the accomplishment of one task under four environmental 
conditions. 


pilot’s task involves a final flare and hover-landing to a 
platform on an oil rig in the open sea. The planform land- 
ing is considered a confined landing area involving the 
need for precision operations to avoid obstructions and to 
properly position the aircraft on the platform. 

To continue this example, assume that three more 
pilots fly the same task under the same conditions and that 
they individually complete a data card. Their findings are 
summarized in figure 5. It is obvious that these four pilots 
did not totally agree, but when we analyze the results, we 
find the data are quite usable. First, we observe that the 
weather is never as constant or homogeneous as we would 
hope. As a result, all pilots probably operated the aircraft 
under slightly different conditions. Second, it is interest- 
ing to discover that pilot Black is most familiar with the 
aircraft and has extensive experience operating from plat- 
forms and ships at sea, day and night. Conversely, Brown 
has the least experience with the aircraft and the task- 
environmental situations evaluated. 

The ratings in figure 6 are then the sum results of four 
pilots evaluating their personal “pilot-machine” perfor- 
mance under four task-environment situations. It must be 
understood that the rating process is personal. It refers to 
the performance that the evaluation pilot has achieved in 
flight. This performance evaluation is then something of a 


self-appraisal and is the product of the pilot’s skill level at 
the time, as well as the personal experience accrued by the 
pilot prior to the flight event that produced the recorded 
pilot rating. 

This is the way the process should work. Some 
flying-qualities analysts ask pilots to establish a rating 
which they feel would reflect how the average pilot would 
evaluate a task. Such an approach is not applicable here. 
For this method to work, pilots must rate their personal 
performance. 

The results summarized in figure 5 have been plotted 
in figure 6. This plot illustrates the preferred data presen- 
tation format for most comparative analyses. The format 
has been designed to be easily understood, and a shaded 
band has been added to figure 5 to emphasize the lack of 
scatter. 

As noted before, there is some scatter in the data, but 
not a great deal. Experience has shown that the scatter 
will increase as the environment becomes extremely 
adverse. A larger scatter band is also possible when pilots 
are asked to evaluate degraded modes that they do not 
have a great deal of experience with. Both situations seem 
to suggest that a lack of pilot familiarity with the task or 
environment can produce scatter. This apparent uncer- 
tainty is both understandable and acceptable. 
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T ASK Normal Flare, Hover-landing onto Confined Elevated 
Platform Area. 

ID 

CODE 

PILOT 


f /f /& / 

A 

2 

1.5 

2.5 

2 


B 

2.5 

2 

3 

2.5 


C 

5 

4 

5.5 



D 

6 

5 

6.5 

5.5 



Figure 5. Summary of pilot assessment data. 


Figure 7 illustrates the next step in the method. For 
this illustration, pilot Green has been asked to evaluate the 
same hover-landing task for three additional and slightly 
different environmental situations (E, F, and G). The air- 
craft is not to be flown specifically to evaluate these situa- 
tions. Instead, the pilot is asked to draw on experience. 
Green can relate well to two of these situations because he 
has personally experienced them in flight. We are not sure 
exactly when, but in any event, he relates well to these 
conditions and is easily able to provide an assessment of 
how well he can fly the aircraft. One situation, G, he has 
not experienced in the aircraft being evaluated, but he has 
flown other aircraft onto similar platforms under 
conditions approaching those identified with G. Thus we 
characterize G as a projected assessment. It is in effect an 
extrapolation. This extrapolation technique is not new; it 
is widely used during early assessments of military air- 
craft, every time development testing is initiated. 

Here again, a certain amount of scatter in the data can 
be expected when the assessments of two or more pilots 
are compared. Projected ratings are subject to the greatest 
scatter, but even that can typically be explained and it is 
normally of little consequence. The scatter in projected 
ratings of operations involving violent weather at night 
can be expected to produce scatter of the order of +2 pilot 
ratings. On the other hand, the data from an extremely 
qualified pilot will often fall along the mean of the scatter 
in the projected data developed by less-qualified pilots. 
The data developed by pilots who do not understand the 


pilot rating process are normally in conflict with the group 
and can be easily identified as such, and discounted. 

Figure 8 illustrates one way that pilot ratings can be 
plotted for analysis. Note that the sets of conditions have 
been ordered across the chart in a way that allows the rat- 
ing to ascend from left to right. This results in a situation 
where the sets of environmental factors are becoming 
more adverse left to right. This arrangement enhances 
data analysis and helps the evaluator insure that a com- 
plete spectrum of task complexity has been considered. 

A simulator can be evaluated by one pilot or by a 
team of pilots. To simplify this next discussion, one pilot, 
Green, will be considered. Remember that the data in fig- 
ure 8 represent the best characterization of the real aircraft 
that Green was able to establish. Assume for the moment 
that the data provided by the remaining pilots would have 
nominally agreed with Green’s data. This confirms that 
Green’s ratings of the seven different operating environ- 
ments is sufficiently accurate to use in the evaluation of a 
simulator. In addition, an inspection of the seven opera- 
tional environments used in flight confirms that they 
probably provide an adequate spectrum of situations to 
use as simulation environments for evaluating a simulator. 
That is, a simulator operator can be asked to electronically 
program the simulator to present the evaluation pilot with 
a set of winds, turbulence, and visual scene factors that 
collectively represent each of the environmental condi- 
tions relating to each of the situations defined in figure 8. 
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TASK SHORT TITLE 

PLATFORM HOVER-LANDING 

PILOT DATA CARD SIM FLT □ 

Pilot Name: GREEN A/C FLT ED 

TASK: Low hover in confined area. Landing on a platform one hundred feet above a water surface. 

Obstructions are present ahead and to the right. Upon landing rotor clearance is 30 feet to 
closest obstruction. Steel structure rises ahead. 

SITUATION 
ID CODE 

FACTORS DEFINING THE 
TASK ENVIRONMENT SITUATION 

PILOT 

ASSESSMENT 

(RATING) 

A 

Clear Day, Calm Air. 

2 



B 

Clear Day, 1 0 KT RT Cross Wind. 

2.5 



C 

Clear Day, 1 0 KT RT Cross Wind, Gusting to 1 7 KT. 

5 



D 

Night, Overcast, no surface lights, single landing LT, 
10 KT RT Cross Wind, Gusting to 1 7 KT 

6 



E 

Night, Full Moon, Stars, Hover Lights, 10 KT RT Cross Wind, 
Gusting to 17 KT. 


3.5 


F 

Night, 1/4 Moon, Single Landing LT, 10 KT RT Cross Wind, 
Gusting to 1 7 KT. 


5.5 


G 

Night, Thunderstorm, 20 KT Wind, Gust to 30 KT. 



7.5 

OPERATING STATE: Normal 

CONFIGURATION: Mid wt, mid C.G., Doorsclosed 

Kg 

UJ > 

GC 0 C 

E uj 
5<2 

PAST 

OBSERVED 

PROJECTED 

Note 1: Tower obstruction lights, landing pad edge lights. 

«g 


Figure 7. Pilot rating card for flight evaluation of an aircraft. 
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APPLICABLE ENVIRONMENTAL FACTORS 

A. Clear Day, Calm Air. 

B. Clear Day, 10 KT RT Cross Wind. 

C. Clear Day, 10 KT RT Cross Wind, Gusting to 17 KT 

D. Night, Overcast, no surface lights, single landing LT, 

10 KT RT Cross Wind, Gusting to 1 7 KT. 

E. Night, Full Moon, Stars, Hover Lights, 10 KT RT Cross Wind, 

Gusting to 1 7 KT. 

F. Night, 1/4 Moon, Single Landing LT, 10 KT RT Cross Wind, 

Gusting to 1 7 KT. 

G. Night, Thunderstorm, 20 KT Wind, Gust to 30 KT. 

MODEL XYZ HELICOPTER 
TASK: 

- LOW HOVER IN CONFINED AREA 

- SEA LEVEL CONDITIONS 





Figure 8. Building a more complete characterization. 


Assume that these situations are simulated one by one 
and that the pilot establishes an assessment (rating) for 
each and enters this rating on a pilot data card as illus- 
trated in figure 9. Now pilot Green has generated two sets 
of ratings trying to accomplish the very same task. One set 
responds to his experience in the real aircraft and one 
responds to his evaluation of the representation of the air- 
craft and visual scene provided by the flight simulator. 

The pilot has in fact rated his ability to achieve a given 


task with a specific degree of precision (performance) at a 
given level of effort. It should therefore be possible to plot 
both sets of data on one chart to determine the degree to 
which the data agree or disagree. 

This has been done and the results are presented here 
as figure 10. Figure 10 shows that the three pilot ratings 
established during “daylight” operations in the simulator 
are roughly two pilot ratings higher than the trend band 
which bounds the data defined for flight in the real aircraft 
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TASK SHORT TITLE 

PLATFORM HOVER-LANDING 

PILOT DATA CARD SIM FLT El 

Pilot Name: GREEN A/C FLT 1 1 

T ASK : Low hover in confined area. Landing on a platform one hundred feet above a water surface. 

Obstructions are present ahead and to the right. Upon landing rotor clearance is 30 feet to 
closest obstruction. Steel structure rises ahead. 

SITUATION 
ID CODE 

FACTORS DEFINING THE 
TASK ENVIRONMENT SITUATION 

PILOT 

ASSESSMENT 

(RATING) 

A 

Clear Day, Calm Air. 

D 



B 

Clear Day, 10 KT RT Cross Wind. 

4.5 



C 

Clear Day, 1 0 KT RT Cross Wind, Gusting to 1 7 KT. 

6.5 



D 

Night, Overcast, no surface lights, single landing LT, 
10 KT RT Cross Wind, Gusting to 17 KT 

6.5 


■ 

E 

Night, Full Moon, Stars, Hover Lights, 10 KT RT Cross Wind, 
Gusting to 1 7 KT. 

D 


■ 

F 

Night, 1/4 Moon, Single Landing LT, 10 KT RT Cross Wind, 
Gusting to 17 KT. 

6 

■ 

■ 

G 

Night, Thunderstorm, 20 KT Wind, Gust to 30 KT. 

n 


zn 

OPERATING STATE: Normal 

CONFIGURATION: Mid wt, midC.G., Doors closed 

CURRENT 

OBSERVED 

PAST 

OBSERVED 

PROJECTED 


Figure 9. Pilot rating card for simulator evaluation. 


during similar conditions. In constrast, pilot ratings 
assigned for simulated night operations are in reasonable 
agreement with the pilot’s earlier characterization of the 
real aircraft. 

On first analysis, these data suggest that the pilot 
found the simulated daylight-visual task to be substan- 
tially more difficult than he found the task of operating 
the real aircraft in the real world. Continuing with this line 
of thought, the increased difficulty is probably a result of 
some lack of authenticity in the visual scene. The agree- 
ment between aircraft and simulator experience at night 
suggests the pilots did not detect any shortcoming in the 
simulator when the simulated scene contained only a 
modest amount of microtexture. That is, the authenticity 
of the visual scene became less important during situa- 
tions in which poor definition was involved. 

The evaluation-charting process can be used to evalu- 
ate the authenticity of flying qualities as well. The data in 


figure 1 1 provide such an example. The data plot indicates 
the real aircraft was much more difficult to fly than the 
simulator. This disagreement in ratings may have been 
caused by simulator control characteristics (being too 
good) or by the simulator model being less sensitive to 
turbulence than it should have been. It is also possible that 
the wind/turbulence model is in error. Regardless, the data 
trends are consistent and have meaning. 

This process can be repeated for (1) failure modes, 

(2) tasks that require gross-aggressive maneuvering, and 

(3) instrument flight where all reference is to cockpit dis- 
plays. The results should allow the evaluation team to 
accurately determine the utility of the simulator. Most 
important, the process will help everyone gain a better 
understanding of the subject aircraft and of the procedures 
and techniques pilots employ during its operation. If 
everyone agrees about the way the aircraft should be 
flown, and if they all evaluate the simulator using these 
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8 | A Observed in Flight Simulator. 
Results of Flight Evaluation. 

7 I D = Day Operations 
N = Night Operations 
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Figure 10. Comparing simulation assessments and real aircraft experience. 


common methods, the evaluation will most likely produce 
results to which most pilot-evaluators will be able to 
ascribe. Agreement in these areas will help preclude 
misunderstandings regarding simulator value and 
applicability. 

Finally, charts should be established for a family of 
flight phases. Failure modes should be examined for each 
flight phase considered to be critical to the crew training 
capability of the simulator. 

A final set of graphics, figures 12(a) and 12(b) has 
been included to illustrate how a real pilot evaluated two 
real but very different aircraft during the accomplishment 
of a real task. Observe in figure 12(a) that the ratings 
dropped from 4.5 for C to 4 for D for the single-rotor 
helicopter, and that there was no change in the pilot’s rat- 
ings for the tandem-rotor helicopter under these two dif- 
ferent environmental situations. This means that, in the 
case of the single-rotor aircraft, the condition established 
by C was more stressful than the condition established by 
D. That is, the crosswind was important to the single-rotor 
helicopter, but insignificant to the tandem-rotor heli- 
copter. In fact, the loss of the crosswind was more impor- 


tant in reducing workload than the loss of daylight was to 
increasing workload. 

Thus the environments should be reordered so that 
they are progressively more severe from left to right. This 
has been accomplished in figure 12(b) and the result is a 
more orderly plot, one which is easier to compare and 
analyze by the general public. 

The scope of this presentation did not allow a com- 
plete treatment of the data collection-presentation meth- 
ods that have been developed by Starmark. I encourage 
you to tailor and expand the concepts presented here to fit 
your individual needs. 

There are many ways to achieve further reductions in 
scatter and ways to determine the importance of a given 
failure mode to the training experience. Many of these 
additional attributes became obvious to the evaluation 
engineer as experience is gained during application of the 
process discussed here. 

Everyone who elects to use this material as a guide is 
encouraged to concentrate on the task of defining the 
combinations of environmental factors that (1) pilots have 
personally experienced and that can best define the 
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Figure 1 1. Data indicating that simulator is too easy to fly. 


normal operating envelope, and (2) allows pilots to feel 
they can also best define the extremes of the operational 
envelope. If the simulation device can provide an 
adequate, authentic training experience under both 
situations, the usefulness of the simulator will have been 
validated in terms of handling qualities and visual scene 
representations. 

MR. WARTH: It is good to see there is a life after 
flying. How close do the ratings have to be to be consid- 
ered a good match in the Cooper-Harper figures? 

MR. GREEN: I am saying when you write down a 
definition or expand the definition to meet your needs, 
just try to keep it in the perspective of Cooper- Harper. 
There are references that you can use. Did I answer your 
question? 


MR. WARTH: How close do the numbers have to 
be? 

MR. GREEN: You mean scattering of the data? 

MR. WARTH: I mean between the simulator and the 
aircraft. 

MR. GREEN: Well, see, that is a whole other discus- 
sion. I think just as a very quick answer, that if you could 
get within a pilot rating and a half, you would think you 
had died and gone to heaven, and you would want it to be 
a little more difficult in the real aircraft, I would guess. 
But what I would do is slip the whole scale to the right. In 
other words, my visual is wrong. I would say my visual is 
wrong or something else is wrong, just as long as we 
don’t give the pilot a misimpression of the handling quali- 
ties of the aircraft, or misinform him somehow. 
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Figure 12. Pilot rating data for single- and tandem-rotor helicopter conducting precision hover, (a) Original sequence of 
environmental factors, (b) reorder sequence of environmental factors. 
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David L. Green is president of Starmark Corporation. He is widely recognized as an 
aviation safety expert and author. Mr. Green is a graduate of the U.S. Naval 
Academy and of the Navy Test Pilot School, and is the author of the first helicopter 
stability and control flight test manual. He managed rotorcraft flight tests for 
Fairchild-Hiller and was vice president of Pacer Systems, Inc., prior to his employ- 
ment at Starmark. Mr. Green has been involved in FAA rotorcraft evaluation projects 
since 1968, and has flown 73 models of helicopters, V/STOL, and propeller and tur- 
bojet aircraft. He is now an adjunct associate professor of aviation systems at the 
University of Tennessee. 
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Since it is late in the day I would like to tell you a use (accident/100,000 departures, risk of serious injury, 

brief story about helicopter safety which was mentioned etc.)- There are a lot of parameters that fixed-wing people 
this morning by Dick Birnbach and a few others, how we use (accidents/100,000 hours, accidents/1 00,000 passen- 

have gotten to where we are. ger miles, etc.). I am limited by time to reviewing only 

I would like to discuss the last 5 years of training and one set of data, and I have accepted the following defini- 

how we have improved and how we have reduced acci- tion; I hope you will, too. “Safety is the identification and 

dents by doing cognitive training. And finally, I would control of risk according to some preconceived 

like to suggest appropriate thoughts for our discussions parameters.” 

tomorrow. Historically, the FAA and NTSB supply data for 

The following is a quote from Dwight Eisenhower. accidents per 1 00,000 hours. The data set shown in fig- 

Like all political quotes, it can be taken in many ways. ure 1 came from Jim McDaniel’s office when we looked 

“Things are more like they are now than they ever have at safety parameters. Accidents per 100,000 departures, 

been before.” It made me think that we haven’t come very accidents within a mile of a heliport, and years between 

far since the workshop in 1985. But you can also look at it accidents in terms of a facility. As you can see, there is a 

as an opportunity to accomplish some things in this work- quarter century of data shown in figure 1 . It tells a very 

shop. I hope by the end of the presentation you will interesting story. At least in the United States you notice 

understand in what way I have contributed to it. in 1965 we were running 55 or 60 accidents per 

Let’s talk about safety and the general definition of 100,000 hours (total fatal and nonfatal). 
safety. There are a lot of parameters that helicopter people 



I960 1970 1980 1990 

CALENDAR YEAR 


Figure 1 . Safety needs. 
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And then over a period of about 10 years, 1965 to 
1975, we dropped by almost two-thirds down to 20. Those 
of you in this country who have been in the industry that 
long realize that that was the time the turbine engine was 
introduced. About 1965 we were almost 100% pistons. 
Then the turbine was introduced, with its higher mechani- 
cal reliability, easier maintenance, and various safety 
improvements. I don’t want to imply that the turbine was 
the only change, but it was one of the major changes that 
occurred during the 1965-1975 period. 

During the middle of the time period covered in 
figure 1 (about 1975), we had a bunch of very experienced 
military pilots returning from Vietnam. Those pilots were 
military and human and they had good and bad habits; 
however, they did have a high degree of experience in risk 
management, which has been mentioned by several peo- 
ple today. They were able to work under high-workload, 
stressful conditions. Later, I will point out some areas 
where simulators may be used to provide more realistic 
risk- management training. 

Then, about 1975 to 1980, in this country we began 
to realize that all of these accidents, at least a large per- 
centage of them — 65% in the entire helicopter commu- 
nity, if you looked at the more high-risk EMS it is nearer 
80% — were all human-error related. The same thing was 
occurring in fixed wing; about 80% of fixed-wing acci- 
dents were also attributed to pilot error. The bottom line 
was to start stressing the human elements in studies. As a 
result, NASA developed a substantial effort in the area of 
cockpit resources management. And we were successful 
in bringing the accident rate down somewhat, although it 


is leveling off as you can see. I think that is the challenge 
we face here. Getting back to the study that generated the 
curve shown in figure 1 , we set a goal of trying to get the 
rate down to 4.5 accidents per 100,000 hours by 1995. 

I would like to talk about a successful human-error 
reduction program and about conventional training and 
some ways we may begin to depart. Figure 2 depicts the 
basic novice pilot, or ab initio, coming in with a lot of 
knowledge. He knows systems, he knows aerodynamics, 
he knows the ATC system, he knows weather, he knows 
procedures on top of that, stall practice, autorotation, 
things like that. He builds skills in flying the aircraft. 

Until recently, 1985-1986, it was always thought he could 
only learn good judgement or decision-making through 
experience. We all know that led to a lot of bending of 
metal and unfortunate injuries and accidents. 

So the FAA set out, between 1975 and 1985, on a 
program to see if we could train and actually teach better 
decision-making in the classroom. It has turned out to be 
very successful, as Pete Hwoschinsky mentioned this 
morning. We generated 15 different manuals, everything 
from students’ private manuals up to manuals for adminis- 
trators and Part 1 35 operators. These have been used 
throughout the industry and the military. 

As an example, Petroleum Helicopter, Inc. (PHI) 
looked at their accident data from 1982 through 1986. 
Correct me if I am wrong, Jerry [Golden], but you fly 
about 2 million takeoffs and landings annually. Before 
1986 PHI could not get the accident rate below two per 
100,000 hours. And in 1987 they dropped it to 1.86 per 
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100,000 hours. The following years, after all the pilots 
were trained, they dropped it to 1.046. 

The Navy did a similar thing and reduced the human- 
error factors in helicopter accidents by 51%. Bell has 
introduced advanced decision-making into their world- 
wide 206 safety seminars. They believe that even though 
they haven’t reached all their operators, they have 
achieved a reduction of 3 1 % in human-error accidents. 

The bottom line is we can train decision-making, but 
there is a problem. The problem is that when we look 
closely at the procedures and the attitudinal training we 
developed, they work much better with the ab initio and 
less experienced (5-year-and-under) pilot. 

The research I am working on now is aimed at how 
we get at the more experienced pilot, how does he think 
differently? At the same time, we were getting all the 
good results in the helicopter community. The air carriers 
were having some spectacular saves or, as Dick Birnbach 
said, some diabolical failures: failures of aircraft materi- 
als, the Sioux City accident, a lost engine, a lost hydraulic 
system. The two decompressions, Flight 81 1 United 
Honolulu, and the Flight 232 Maui accident Aloha — both 
aircraft incurred very large holes in the fuselages. 

In the case of 232, 1 will just dwell on two of the suc- 
cesses for a minute. The captain had access to a training 
airman in the back who know how to control pitch and 
yaw with the throttles. The captain immediately accepted 
his volunteer and used him to control lateral movement 
and aircraft pitch attitude. While they were doing that and 
checking to make sure that passengers were prepared for 
an emergency, they were still fighting a tendency for a 
38° right bank and severe pitch oscillations, or phugoids. 
Nevertheless, as you know, they successfully brought the 
aircraft down, at least in a partial save. 

Both of these decompression accidents (UAL 
Honolulu and Aloha, Maui) are very interesting because 
the pilots and crew acted contrary to handbook training 
procedure, which would have had them dive to regain 
cabin pressure. The captain decided that would be a bad 
move, because it might enlarge the hole in the fuselage. 

As a result, he decided to slow the aircraft. However, he 
didn’t know the speed at which the aircraft would stall, 
given the big hole in the fuselage and the extra drag it 
created. 

The second important thing about all of the saves — 
all the time they were handling the emergencies while 
creating new procedures, if you will, in response to the 
cues they had. They were able to keep up the housekeep- 
ing chores, they communicated with ATC, they did engine 


shutdown checklists, all the things they were trained to 
do. That is a lesson we will get back to in a minute. 

During the past 30 months I have been looking at the 
accidents, looking at the difference between experts and 
ab initios, and it turns out there are 24 different character- 
istics that distinguish experts from novices. I have sum- 
marized the top five in table 1 . Believe it or not, in an 
emergency pilots go back to what their instructors told 
them — they fly the airplane. That is evidenced in all the 
accidents studied, they have instantaneous recall of train- 
ing; in some cases it takes on the characteristics of 
instinct. They maintain their composure, they come up 
with a reasonable plan, and they execute it with all their 
available resources. It is not surprising that this is exactly 
what we have been trying to train for with the cockpit 
resources management program. Finally, as we know, 
pilots are goal-oriented, self-assured individuals. 

Table 1 . Training needs: expert characteristics 


1 . Reversion to basic airmanship skills 

2. Instantaneous recall of training 

3. Reasoned approach in emergencies 

4. Positive in approach and expectations 

5. Self-assured and optimistic 

I would like to look now at a few of what we call 
fatal fallacies (table 2). They are attributed to Dr. Walt 
Schneider at the University of Pittsburgh. He looked at 
both air-traffic control and aviation accidents and came up 
with these six fallacies. I don’t know why he termed them 
fatal, but undoubtedly he has his reasons. 

Basically, practice makes perfect is a fallacy because 
it is a bump and grind approach. It does work, but it does 
not have a lasting effect on most people. In some cases the 
procedure is never learned properly. Training a task in 

Table 2. Training needs: fatal fallacies 3 


1 . Practice makes perfect 

2. Train in the form to be used 

3. Skill training is intrinsically motivating 

4. Must include high accuracy standards 

5. Initial performance predicts eventual outcome 

6. Intellectual understanding produces proficiency 

a Dr. Walter Schneider, University of Pittsburgh 
Learning and Development Center. 
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exactly the form in which it is to be used is time-ineffi- 
cient. We talked about autorotation earlier, sling loads, 
things like that. In the fixed-wing, we having holding 
patterns. All those things can be learned much better and 
retained better with quick reenforcement practicing, 10 or 
20 an hour as opposed to 1 or 2 an hour. They are things 
that can best be done in a simulator. 

Number 3 — skill training is intrinsically motivating — 
is interesting because flying is fun in itself and people are 
motivated to learn how. But even though that might be 
true initially, after you have been at it for 5 years it seems 
that the basic thrill is usually gone and you are going 
through the hoops, going through the FAA-required 
checklist of maneuvers. But again, what they found at 
Pittsburgh was that if they had bells and interesting 
sounds and visual cues for training reenforcement, they 
had a 30% to 50% reduction in failure rate. 

The fourth one-— high accuracy standards — is particu- 
larly pertinent. That is, we all think about high accuracy 
standards — good steady needles, good heading and alti- 
tude control; these are very important, especially in the 
real world. But they are not necessarily the best way to 
train in a simulator. What happens when you become a 
very accurate, precise pilot? You may not be very good at 
other things, like high-workload tasks, emergencies, 
multiple-tasking, sharing your attention. These are best 


taught in a loft scenario, in a simulator, in composite high 
stressful situations, as Dave Green said in his presentation 
earlier today. 

The last two of the fallacies are self-expanatory. 

Early this morning I heard some words from the FAA 
that got me very excited. The regulations are being 
changed to allow the inclusion of more simulators. What 
is appropriate training? What can we do? What should we 
do in simulator versus aircraft? I submit that the current 
standards for simulator uses (table 3), though limited, 
should be retained and should not be thrown out with the 
bath water, as someone said. Greg McGowan pointed out 
that he trained nearly 10,000 pilots with these; the evi- 
dence I pointed out earlier documents that it works, as 
well. My perception was that at least part of the reasons 
for the four required aircraft maneuvers was that the FAA 
needs to maintain control. I again need to suggest that we 
have to discover whether the hover and the current four 
maneuvers are the correct ones to retain. 

Ed Boothe suggested an exemption if somebody 
wants to come up with that. I think in this group, with the 
expertise we have, we can come up with a better set of 
criteria. 

As far as interim uses are concerned (table 3), I have 
been thoroughly brainwashed by Curt Treichel and others 
in this room to think that if a pilot has the experience, 


Table 3. Appropriate training 


1 . 

Current simulator uses 



Biannual flight review 

61.56 


D/N currency, instrument competency 

61.57c,d,e(2) 


12/24 month PIC check 

61. 58b, c 


ATP rotorcraft type check (90%) 

61.163a 


Initial/recurrency testing 

135.293 


PIC instrument proficiency 

135.297 

2. 

Quality control 

Hover requirement versus hover proficiency 
Emergency procedures (discussion vs experience) 


3. 

Desired near-term uses 
FAR approval versus exemption 



ATP rotorcraft add-on type rating 

61.163a 


Commercial add-on instrument 

61.65g 


ATP airplane add-on rotorcraft category 

61.165 


Instrument instructor 

61.191 

4. 

Alleviate training fallacies 


5. 

Support overall training and licensing system 
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commercial rating, ATP fixed wing, there is no reason he 
can’t get the helicopter ATP add-on in the simulator. 

And at the top of that list, I think we all would like to 
see FAR approval of simulators as opposed to the timely, 
costly exemption process. I think again, that together we 
can come up with scenarios and lists of tasks that can 
alleviate training fallacies. I am not talking about turning 
things upside down that we have today, but about just 
looking at the real world. 

Finally we need to come up with an integrated 
approach. I haven’t heard anybody come up with a sys- 
tems approach from the top down to designing a training 
program. Far-term or blue sky, more controversial might 
be total licensing and testing in the simulator (table 4). 

We would like recognition of helicopter simulators equal 
to that granted the fixed-wing simulators. There is no rea- 
son that if a 727 pilot can get his type rating in a simulator 
that we can’t get type ratings in an S-76 simulator some 
day. I don’t know how many of you have looked at the 
ATP program, which allows trading off simulator time. 
We ought to set our sights on the rotorcraft community 
the next time we are talking about that for helicopter 
simulators. 


Table 4. New frontiers: far-term suggestions 


1. Initial licensing and testing 

2. Equal recognition with airplane standards 

3. Advanced qualification program 

4. Crew testing and licensing 

Finally, there is crew testing. I went over fixed-wing 
accidents and how the interpersonal skills of the crews 
were involved. You can’t test that in an S-76 or in any 
helicopter today. There is no place for the examiner to sit 
back and evaluate the crew. It can only be done in a simu- 
lator. Right now Curt Treichel tells me that crew evalua- 
tions are limited because the test pilot sits in the left seat. 

They think pilot in command (PIC), so we need to 
work on that a little bit. Getting back to our expertise and 
my current efforts. I think there is also an opportunity to 
introduce some new concepts there. When we talk about 
the next generation of decision-making training, the ques- 
tion is when to do it. 

The ab initio pilot knows all the facts; he has the facts 
he needs to know to fly the airplane and to survive 
(fig. 3). The low-time pilot knows how to survive, he has 
instantaneous recall of what to do if the engine quits. But 


MORE EXPERIENCE 
AND TRAINING 



NOVICE LOW TIME EXPERT 

PILOT PILOT PILOT 


INCREASING DECISIONAL SPEED AND ACCURACY 



KNOWING 

1 

1 

KNOWING 

1 

1 

KNOWING 


WHAT 

1 

HOW 

1 

1 

WHEN 

1) 

KNOWLEDGE 

! 1} 

SCHEMA BASED 

Jl> 

CONTEXT DEPENDEN1 


BASE SEARCHING 


PROCEDURES 


ACTION 

2) 

DOMAIN GENERAL 

! 2) 

DOMAIN SPECIFIC 

! 2 > 

EXPERIENCE BASED 

3) 

TIME CONSUMING 

3) 

FASTER 

3 > 

INSTANTANEOUS 

4) 

ACCURATE 

; A, 

MOSTLY ACCURATE 

4) 

SELF-REGULATORY 


Figure 3. New frontiers. 
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he does not know when to alter those actions; he does not 
have the ability to react to novel things. Like the pilot in 
the cabin of Aloha 737, who looked back and saw blue 
sky. I don’t think we can take a true ab initio pilot and 
bring him to that level; it is all in the procedural knowl- 
edge base and how we use the procedures we have learned 
in combination with the knowledge we have and facts that 
we have learned. 

And finally, the expert pilot does all this in a self- 
regulatory mode. Self-regulatory means the next step in 
situational awareness. As I said, the expert can undergo 
an untrained-for emergency like those discussed and still 
maintain his housekeeping chores, carrying out his normal 
ATC communications and things like that. So they are not 
impossible tasks; it is just going to require some new 
training scenarios. 

Finally, the most exciting new frontier I can think of 
is our being here at this workshop and that we have been 
invited to help the FAA generate new standards: 


1 . Joint industry-government simulator qualifica- 
tion standards development 

2. Appropriate and sufficient training and testing 
criteria 

3. Mission- and task-driven qualification standards 
1 think it is a great opportunity and I think from talking to 
the FAA people here, that it is going to be more pervasive 
than just in the simulator area. I welcome the chance to 
work with them. I think we should all think about the 
words “appropriate and sufficient training.” I think we 
have a lot of components, we all know some of the 
weakness. I think I have an idea of some of the new ones 
based on research I have done. We need to think about 
missions and tasks to use for training concepts. 

Thank you for your time and I look forward to work- 
ing with you in the next couple of days. 



Richard J. Adams, vice president of Advanced Aviation Concepts, has worked in 
civil aviation research and development for 26 years. He is the author of 69 technical 
reports, articles, and papers dealing with flight safety, decision-making training, 
pilot-error accident data analysis, air-space/route design, helicopter performance 
modeling, and helicopter pilot training deficiencies. Mr. Adams received a B.S. in 
aeronautical and astronautical engineering from the University of Illinois, and an 
M.S. in mechanical engineering from the University of Florida. Mr. Adams is a regis- 
tered professional engineer in Florida and a private pilot. 
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13. TRANSFER OF TRAINING AND SIMULATOR QUALIFICATION 

OR 

MYTH AND FOLKLORE IN HELICOPTER SIMULATION 


JACK DOHME 


I noticed something yesterday — perhaps others in the 
audience did too. Have you seen any young kids come up 
here and address this body? It would be, perhaps, impolite 
to note that gathered here are the grand old men of the 
field, that is, considerable experience is represented. 

So... I got to wondering why Bill Larsen asked me to 
speak. I’m not a test pilot; I’m not a graduate engineer; I 
don’t have 10,000 hours experience beating the air into 
submission. However, it occurred to me that I wear trifo- 
cals so I’m certainly not a kid. I carry an AARP 
(American Association of Retired Persons — minimum age 
50) card in my pocket so I guess I’m old enough. And we 
do have a perspective at “Mother Rucker” (The Army 
Aviation Center at Fort Rucker, Alabama) that may be 
worth sharing. So, I changed the title of my presentation 
this morning to “Myth and Folklore in Helicopter Simula- 
tion.” This presentation has a second author, my boss. 
Chuck Gainer, and I should note that he contributed ideas 
but did not suggest the mode of this presentation. In 


summary, I suspect the real reason that Bill Larsen asked 
me to address this august body is for comic relief. 

Table 1 is intended to present some political stuff, to 
stir up trouble and to get people to think about the issues 
in that X-rated document we’ve been asked to read, the 
“AC 120-XX.” I thought I would begin by listing three of 
what I’m calling “myths” in the field of helicopter simula- 
tion. In discussing these myths, the “straw man” that I’m 
attacking wears green and I think that’s fairly safe in this 
audience. 

Let’s look at myth 1 : “A Simulator Should Look, 
Taste, and Smell Like a Helicopter.” An IP (Army 
Instructor Pilot) once kidded me that, “If it don’t smell 
like JP-4 (jet fuel), it couldn’t be no good.” Well, what is 
the objective of simulation? Is it to look, taste, smell, and 
feel like a helicopter? Let me answer that question with 
three examples: The Crew Station Research and 
Development Facility (CSRDF) at NASA Ames is an 
engineering simulator, right? Could you train somebody 


Table 1. Myths in helicopter simulation: myth 1 


Simulator should look, taste, and smell like a helicopter 
Must determine the objective of the simulation 
Crew station design/man-machine interface; CSRDF 
Combat training technology/user requirements: SCTB 
Primary training technology/train neophytes: UH-1TRS 
Must define fidelity to meet the objective 
AGARD Working Group (Key 1980) 

1 . “Objective fidelity” - simulator reproduces measurable aircraft states or conditions 

2. “Perceptual fidelity” - degree to which Ss perceive the simulator to duplicate aircraft states or 
conditions 


STI definition (Heffley et al. 1981) 

“Simulator fidelity is the degree to which characteristics of perceivable states induce correct psychomotor 
and cognitive control strategy for a given task and environment” 
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in it? Sure you could, but that’s not its purpose. It was 
created to address questions in crew station design. The 
Simulator Complexity Test Bed (SCTB at ARI, Fort 
Rucker) is a $24 million toy that is coming to Fort Rucker 
this year. Initially configured as an Apache, it has red- 
station/blue-station training capability beyond any heli- 
copter simulator in existence. It is an ideal device for 
developing advanced combat training. It is a trainer, but it 
is more of a training research tool. It is not an engineering 
simulator, not directly. 

Moving from the sublime to the ridiculous, how 
about “Cheap Charlie,” the UH-1TRS? It is a trainer, pure 
and simple. You cannot start it, you cannot fly an ILS 
with it, it does not use fuel, that is, we don’t currently 
drive the fuel gauge. But, it trains “hands and feets,” neo- 
phytes, kids off the street. In other words, it has evidenced 
significant positive TOT (transfer of training) to the UH-1 
aircraft using neophyte pilots as research subjects. I think 
we should keep the objective of a given simulation in 
mind as we review our ideas today. 

Once we have decided on the objective of a given 
simulator, an associated issue is the question of simulator 
fidelity. I brought some of my favorite definitions that I 
think are worth reviewing (table 1). Dave Key, who was 
in the audience yesterday, was the key player, no pun 
intended, in the AGARD working group in 1980, when 
they distinguished between “perceptual versus objective 
fidelity.” The issue here is, do you want to measure what 
the simulator does and compare it with the aircraft, or do 
you want to measure what the “bus driver” does and com- 
pare pilot responses from the simulator to the aircraft? I 
think the latter is more appropriate, at least from a 
trainer’s perspective. 


The definition we most commonly use at ARI is that 
set forth by Heffley and a cast of thousands at Systems 
Technology Incorporated (STI). STI did a report for us 
that defined fidelity as “the degree to which characteris- 
tics of perceivable states induce correct psychomotor and 
cognitive control strategy for a given task and environ- 
ment.” Although I worry about the word “correct,” I think 
this definition is worth considering; it focuses on the bus 
driver and not on the bus. 

While we are reviewing the issues involved in simu- 
lator fidelity, I think it is worthwhile to reconsider Vernon 
Carter and Clarence Semple (table 2). When I first read 
their definition of “error fidelity,” I thought, what kind of 
nonsense is that? Any good psychologist knows about 
error-free learning. But then, I thought the definition and 
saw that it has several important advantages. Looking at 
the error distribution that students make in a simulator and 
in the aircraft places the focus on the behavior of trainees, 
with the ultimate goal being “good” performance in the 
aircraft. Although this definition is specific to training 
simulators and not engineering simulators, it does suggest 
a metric for simulator evaluation... training errors. 

At the bottom of table 2, I’ve included a reminder 
from Ed Eddowes and Wayne Waag: “There is no com- 
pelling relationship between training effectiveness and 
fidelity /realism.” That’s the kind of statement I’d like to 
use as a final examination question. We could ask the 
students to react to it as either true or false and then write 
a short essay to support their choice. The students could 
get 100% credit for agreeing or disagreeing, depending on 
the strength of their arguments. I think I would disagree 
because training effectiveness is a practical definition of 
fidelity. If a simulator trains, it has fidelity. . . who cares 
what it looks like? 


Table 2. Myths in helicopter simulation: more myth 1 


Carter and Semple (1976) 

“Error fidelity” - assumes objective is training 

1 . Trainees make same errors in simulator and aircraft 

2. Relative frequency distribution of errors same in both simulator and aircraft 

3. Effect of trainee errors on system performance is same in both simulator and aircraft 
Advantages of concept 

1 . Focus on behavior of trainees 

2. Recognizes ultimate goal - performance in aircraft 

3. Suggests a metric - training errors 
A reminder (Eddowes and Waag 1980) 

“There is no compelling relationship between training effectiveness and fidelity/realism” 
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Table 3 suggests a second myth: “The engineering 
test pilot knows best.” In this myth, the bad guys are not 
people like Roger Hoh, they are the green-suited simula- 
tor test pilots. We all know how the Army goes about 
accepting a helicopter simulator for training. There’s a 
procedure called Operational Test-2 (OT-2) in which a 
would-be expert, usually a senior warrant officer with a 
lot of time in the airframe, is assigned to make subjective 
judgments regarding the simulator’s handling qualities. I 
don’t necessarily mean Cooper-Harper ratings but some- 
thing more subjective than that. Then the software is 
“tweaked” to satisfy the judgment of the “expert pilot.” 
This is the way simulators are accepted into the Army 
inventory. 

Is there anything wrong with this approach? Yes 
there is! The smart folks at STI, Hogue, Jex, and 
Magdelano evaluated the Army’s UH-60 simulator. The 
UH-60 simulator has a six degree of freedom (DOF) syn- 
ergistic motion base, but the STI report noted that as a 
result of the OT-2, two of the degrees of freedom were 
“tweaked” entirely out of existence! Specifically, the sim- 
ulator has only pitch, roll, yaw, and heave. It has no 
measurable sway or surge. The Army owns 18 UH-60 
simulators, 17 in the field and one at the factory in 
Binghampton. And none of them exhibits more than four 
DOF. Is that what improving simulator fidelity is all 
about? It doesn’t make sense to me. But, if we’re going to 
attack this green straw man, let me offer an alternative. 

Yesterday, Ken Cross (Anacapa Sciences) offered 
“backward transfer” as an empirical yardstick with which 


to evaluate existing simulators. Senior aviators performed 
emergency touchdown maneuvers in the AH-1 Cobra 
aircraft until they met published criteria. Then they flew 
the same maneuvers in the AH- 1 flight simulator: 58% 
failed one or more maneuvers. The backward transfer 
ratios were relatively low, ranging from 0.16 to 0.43. 

Since the aviators had been qualified in the aircraft within 
the past few days, it is unlikely that they “forget” how to 
accomplish the maneuvers. It is more likely that the skill 
requirements in the simulator and the aircraft are not the 
same. As Ken Cross noted, the existence of positive TOT 
data does not necessarily mean that the simulator is effec- 
tive. The OT-2 report on the AH-1 simulator (by Bridgers, 
Bickley, and Maxwell) cited some evidence of positive 
transfer to the aircrarft and yet look at the results of the 
backward transfer study. Positive TOT alone may simply 
reflect some procedural transfer to the aircraft while 
obscuring a substantial aerodynamic deficit that will limit 
the overall training efficacy of the simulator. 

Can we improve on the subjective pilot opinion 
method of evaluating a simulator’s effectiveness? I think 
so. Let's look at Stan Roscoe’s transfer effectiveness ratio 
(TER) (table 4). As an example from our Cheap Charlie 
research, we took a random sample of 10 Army officer 
trainees and dragged them kicking and screaming into the 
UH-1TRS where we substituted 9 hours of simulator time 
for 9 hours of aircraft time. We trained them to published 
criteria in the simulator (three successive maneuver itera- 
tions that met the Flight Training Guide standard) and 
then we employed the same criteria on the flight line in 


Table 3. Myths in helicopter simulation: myth 2 


‘The engineering test pilot (or SIP or Eagle Scout) knows best” 

Army acceptance test procedure 

1. Assign an “expert” 

2. Subjective judgment of handling-qualities/training features 

3. “Tweak” the software 

Outcome (Hogue, Jex, and Magdaleno 1982) 

1 . UH-60FS has six DOF synergistic motion base 

2. Only four DOF (no sway or surge) 

3. Army has 17 fielded UH-60FSs with four DOF motion bases 

Alternative approach: empirical yardstick to evaluate existing device - backward transfer 
Example (Kaempf and Blackwell 1990) 

1 . Trained to criterion in AH- 1 Cobra (ETMs) 

2. Flew AH-1FWS: 58% failed one or more maneuvers 

3. Backward transfer ranged from 0. 16 to 0.43 

4. Demonstrates skill requirements different in aircraft and simulator 
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the aircraft. We compared them with a control group of 
students who did not have simulator training. I’ve 
included a kind of “middling” example, traffic pattern 
flight. We found that the control group required about 
21 maneuver iterations to meet the standard whereas the 
experimental (simulator-trained) students required about 
13. That savings of about eight maneuvers on the flight 
line can be divided by the “cost” of producing the savings: 
about 13 iterations in the simulator. Thus, the TER for 
that manuever is 0.60. This could be interpreted as mean- 
ing that the simulator was about 60% as effective as the 
aircraft, using the aircraft as the criterion measure. This 
metric has the advantage of measuring “in vivo” training 
effectiveness of actual flight students embedded in the Ini- 
tial Entry Rotary Wing (IERW) training program. 

Let’s look at the final myth: “The more features the 
better” (table 5). Here, we tread on some hallowed 
ground. My favorite example, after 14 years at Fort 
Rucker, is in the area of motion-base requirements. I 
recall the Singer-Link folks telling the Army that the cost 
of a simulator motion base adds only 2% to the total 


device cost. To evaluate that assertion, I’d like to develop 
ROC (Required Operational Characteristics) requirements 
for the LHX simulator specifying no motion base and 
then, on the day of the best and final offer, add 2% to the 
contract and say we changed our minds! 

Anyway, the draft Advisory Circular 120-XX that 
Dean Resch and I talked about requires a motion system 
for acceptance, even for level A. Is there any evidence 
that motion even contributes to training, let alone is 
required for training? We’ ve done two small-number 
empirical evaluations at ARI using neophyte trainees, one 
in 1984 using five students on motion and five without 
motion, and one in 1990, with six on motion and six off. 
All students were strapped in the simulator; we erected 
the motion base in every case and students were not 
informed (nor did they guess) that we were evaluating the 
effects of motion on training. In both experiments, the 
nonmotion students outperformed the motion students. 
Now that evidence only pertains to Army ab initio (that’s 
a Latin phrase for “kids off the street”) trainees learning 
basic hovering and traffic pattern skills. However, our 


Table 4. Myths in helicopter simulation: more myth 2 


To evaluate developing technology: transfer effectiveness ratio (TER) 
Measure transfer of training “in vivo” - embedded in training program 
Random sample of trainees 
“Blind” evaluation of flight line - same criterion 
Calculate TER: 

TER = (C a - E a )/E s 

Example from UH-1TRS - traffic pattern: 

TER = (20.7 - 13.2)/12.6 = 0.60 


Table 5. Myths in helicopter simulation: myth 3 


“The more features the better the simulator” 

Motion base - the 2% myth 

Draft AC- 120-XX requires a motion system, even for level A 
Small N research suggest motion may inhibit training 
Instructional support features - unused/unusable 
Auto co-pilot 
Auto check ride 
Recorded demonstrations 
AAA reviews (1982, 1985) 

Insufficient training data to justify acquisitions 
Recommended training requirements - empirical basis 
^^Wentifiedno^bladehou£savings^^^^^^^^^^^^^^^^ 
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research agrees with the literature in finding no significant 
training advantage for a motion base. 

Another example would be Instructional Support 
Features (ISFs). I’m short on time and won’t discuss these 
but table 5 lists three examples from the 2B24 Huey 
instrument flight simulator that either don’t work, are 
virtually never used, or have been recently taken off-line 
by the Army. Couldn’t we have based the simulator fea- 
tures on a research evaluation of the requirements instead 
of just buying all the bells and whistles the manufacturer 
could offer? 

My third example of simulator features requires that I 
bend logic a bit. In 1981, the Army Audit Agency (AAA) 
came to Fort Rucker and evaluated simulator utilization. 
Their 1982 report noted that the written premise for 
procuring flight simulators had been “blade hour savings." 
The folks from the AAA looked around Rucker and 
couldn’t find the money! The Command Group’s answer 
was that there was no intent to reduce flight hours but that 
simulators were training multipliers. There’s nothing 
essentially wrong with viewing simulators as adjuncts to 
“blade hour” training, except perhaps the inherent dishon- 
esty. The AAA made two recommendations: first, that 
Fort Rucker needs more training data to justify further 
simulator acquisitions and second, that something as 
expensive as Army aviation training should have an 
empirical basis. Actually, the AAA said that Fort Rucker 
can have simulators to experiment with in “the school- 
house” but that procurements of simulators for the field 
would be carefully scrutinized for appropriate training 
requirements analyses and for empirical means of estab- 
lishing simulator effectiveness. I think it’s embarrassing 
to have a bunch of auditors come around and tell the 
trainers how to do their business. But it makes the point 


that simulators should be designed, evaluated, and pro- 
cured for effectiveness and not for a bunch of “gee whiz” 
features. 

So, what would we propose as an alternative? Again, 
if you want to stir up a hornet’s nest, you’d better have a 
bug bomb. The philosophy behind our suggested approach 
is to do a thorough, boring, tedious front-end analysis to 
determine the training requirements based on the ultimate 
criterion of mission readiness in the field (table 6). At an 
initial level, that’s not all that difficult to accomplish since 
the Army’s Directorate of Evaluation and Standardization 
(DES) sends flight-skill evaluators worldwide for no- 
notice evaluations. Thus, it should be relatively easy to 
determine where the basic mission-readiness training 
deficits are. Then, a cost-of-training-effectiveness- 
analysis (CTEA) could be used to compare the training 
cost of simulation, blade time, or a combination. In 
summary, if you don’t have a problem training the 
maneuver or mission in the aircraft, don’t design a 
simulator to train it. 

We can also try to design our simulators to be more 
flexible... to anticipate future requirements. We don’t 
want to perpetuate the Army way: procure by publishing 
requirements, discover that the requirements won't get the 
training job done before the device is even fielded, initiate 
a Product Improvement Program (PIP) to modify the 
device to do what you originally intended (but didn’t ask 
for). The PIP system makes the Army look dumb and the 
contractors look wealthy. We should be able to do better. 
Can’t we develop requirements with an eye to the future? 
Can we design part-task trainers and modular simulator 
designs in place of plenary simulators that are designed as 
aircraft replacements? Can we do CTEAs to estimate the 
effectiveness of simulators before we buy them? 


Table 6. Myths in helicopter simulation: more myth 3 


Alternative philosophy 

1 . Perform a front-end analysis 

Training requirements: assess mission readiness in field 
CTEA: Compare aircraft and simulator efficiency 

2. Design/construct modular simulations - flexibility to meet changing requirements 
Design for spare capacity - hardware/software 

Use TOT evaluations of training effectiveness; iterate design 

3. Design/construct part-task trainers instead of plenary simulators 
Design to meet training requirements 

Iterate design 
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So let me go to my last point. In our shop, we call our 
simulator Cheap Charlie because we don’t want to be 
taken too seriously, but also because we want to empha- 
size that it’s a low cost training tool and not a surrogate 
aircraft. In a similar vein, I’ll call our approach the ACME 
“Fly by Night” Simworks to try to keep our attention 
directed to doing useful and meaningful research related 
to our charter; low-cost entry level helicopter flight train- 
ing (table 7). I apologize for the pedestrian acronym, 
ACME, but it may serve to keep our attention focused on 
our research goals. Perhaps it has value to other simulator 
designers, researchers, and users as well. 

Table 7. Myths in helicopter simulation: still more 
myth 3 


ACME “Fly-by-Night” Simworks and Oyster Bar 
Analyze - does it meet requirements? (CTEA) 
Combat - does it address Army mission? 
Modular - is the design flexible? 

Evaluate - does it train? (TOT) 


My wife and I don’t watch much television but we 
have come afficionados of the network show, Twin Peaks. 
My hero, Special Agent Cooper, has a new enamorate...a 
woman recently released from a convent, that is, an ex- 
nun. Given her status, he decided to woo her with a joke 
about penguins. There were two penguins on an ice flow 
in Antarctica and one turned to the other and said, “You 
look like you’re wearing formal evening wear.” The other 
penguin said, “Maybe I am.” The connection to fidelity in 
simulation is obvious, right? 

MR. DAVE GREEN: Just a quick observation with 
which you can agree or disagree regarding your comments 
about motion. I think what we say is that bad motion is 
worse than no motion. When somebody tweaks a machine 
to make motion, it was probably pretty bad motion. When 
you get the kind of training you get by taking motion out, 
it is because motion was a negative training feature. 
Would you agree or disagree? 

MR. DOHME: Well, I would pass the baton. The 
question is, regarding our getting worse training with 
motion than with no-motion, Mr. Green is saying that the 
issue is probably that bad motion is worse than no motion. 
I would agree that perhaps bad motion is worse than no 
motion at all. However, we probably had a most thorough 
evaluation of the motion system on the UH-1TRS by the 
University of Alabama Flight Dynamics Laboratory 
(FDL). The FDL engineers analyzed and tweaked our 


motion system and wrote a thorough report on their 
efforts and I would refer you to that report since I’m not 
an engineer. 

The FDL engineers were convinced that our motion 
base was doing as well as it could, given the limitations 
inherent in simulating the motions of flight. For example, 
the issue of washout. Is it subliminal or not? It wasn’t that 
we were naive regarding the issue of motion base fidelity, 
and we did have simulator-experienced engineers develop 
and tweak our motion-base equations as best they could. I 
would be happy to provide a copy of the report; I think it 
was done right. 

MR. FRANK CARDULLO: I would like to follow up 
on that comment a little bit further. Virtually every 
transfer-of-training study that has been done about motion 
has indicated that there has been no transfer. Unfortu- 
nately, though, just about every transfer study on training 
of motion has been done on bad motion systems. You 
admitted yourself there were two degrees of freedom 
missing. 

MR. DOHME: No, not on the motion system we used 
with the UH-1TRS. All five were working; as a matter of 
fact, we had sway, which, it turns out, the original 2B24 
doesn’t have. 

MR. CARDULLO: But, nevertheless, that one is a 
fairly archaic motion system and the performance is poor, 
and the cueing-out rhythms are poor. That has been virtu- 
ally true of all the motion-transfer-of-training studies. I 
think good motion-transfer-of-training studies should be 
done, and I wish the impetus would come from the Army 
or from your organization in particular to do a good 
transfer-of-training study on a good motion system. 

MR. KATZ: Good suggestion, Frank. I am not here 
to comment on the work that the laboratory did previ- 
ously, but again, along this same vein, because obviously 
your talking invites these comments, let me first of all 
note that you did not say anything abut the effect of 
motion on backward transfer. And you see you had the 
problem of backward transfer, I assume, with motion. 

DR. DOHME: Yes. 

MR. KATZ: And then you had a problem with for- 
ward transfer with the motion so it invites the hypothesis 
that the bad motion as a matter of fact caused this. And 
the thing that I think ought to be studied is to see if the 
backward transfer would also improve by eliminating this 
motion. And then I would make the hypothesis that if you 
get your engineering work up to the level where the 
backward transfer would be good with the motion that in 
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this case also the forward transfer would be good with 
that motion. 

MR. DOHME: Interesting hypothesis. Those, of 
course, are different vehicle we used for those two studies. 
The backward-transfer work was done in Germany. It’s 
difficult to do that kind of in vivo testing in an active 
military unit, but it is a good idea: A motion versus no- 
motion backward transfer study. 

MR. GERDES: My background is about 25 years of 
simulation at Ames, ever since we had first fixed-base and 
then motion-based simulators. And I have extensive expe- 
rience on our five, six and three degree-of-ffeedom simu- 
lators. I’m only saying this to give you some qualifica- 
tions for what I am about to say. 

First of all, I agree very, very highly that no motion is 
better than bad motion. That is what we have been saying 
for years. Second, motion comes into play or is useful in 
an engineering simulator, perhaps more so than in a 
training simulator, where you are looking for, say, the six 
and one half boundary, the boundary where controllability 
or emergency control of the aircraft or helicopter is impor- 
tant. Then motion feedback to the pilot is extremely 
important for the engineering pilot to assess what the 
control problem might be. Third, about five years ago, I 
participated in a simulation on our VMS, which has plus 
or minus 30 feet of vertical travel. It is a six degree-of- 
ffeedom and we did an autorotation simulation. I think it 
was for this particular theme we are looking at, but for the 


Army; in other words, are simulators useful for training? 
And the autorotation maneuver was critical, extremely 
hard to perform and learn and so forth. That one simula- 
tion was probably the one that stands out most in my mind 
as to where motion, and it was good motion, played a 
very, very important part in this training business. 

I was able, with practice, to make a whole series of 
autorotations down to a fairly reasonable area and this is a 
vertical motion simulator. So you have this stress that 
others here have talked about. There is a simulator you 
can break, so you try very hard. With the sound system 
we had, we were able to give the pilot cueing for the rotor 
sounds. When we pitched up to flare, we got the motion 
travel to give us the deceleration and we had to doctor up 
the visuals a little bit. We had to put in a couple of vertical 
towers for visual height perception. 

We did have a fourth window, a chin bubble, we 
could see through as you could on a Huey. We could do 
some fantastic things as far as accurate touchdowns are 
concerned. This was not training, this was an engineering 
simulation in which we varied disk loading, weights, 
winds, all of these things. We did a whole matrix of 
autorotations under difficult conditions, and all of them 
turned out really well and defined boundaries and so forth. 

I am saying all this because motion, when properly 
used, is very good for training, as well as for engineering 
simulations. 



Jack A. Dohme has a B.A. degree in psychology from Baldwin-Wallace College, and 
M.S. and Ph.D. degrees in experimental psychology from the University of Arizona. 
He has taught undergraduate psychology at the universities of Arizona and Maryland 
and at Concordia College and Grinnell College, and has worked as a senior research 
consultant at Stanford Research International. His primary research interests are avi- 
ator training, flight simulation, and human performance measurement. Dr. Dohme is 
a licensed fixed-wing and helicopter pilot. 
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14. VALIDATION AND UPGRADING OF PHYSICALLY BASED 

MATHEMATICAL MODELS 


RONALD DU VAL 


I appreciate this opportunity to talk to this select 
group about these issues. The subject is validation and 
upgrading of physically based mathematical models. 
There are a lot of terms that are going to have to be 
defined. 

The previous speaker discussed validation from a 
totally different standpoint from the one I am going to 
address. He was looking at total validation of the simula- 
tion complex, which involves the motion-based system, 
the visual system, the transport delays — everything that 
affects the way a pilot perceives what is going on in the 
simulator. The starting point for all of these issues, how- 
ever, is the mathematical model that drives all of these 
systems. And it is very difficult to determine what consti- 
tutes validity in terms of visual display or what constitutes 
validity in terms of motion-based display. 

On the other hand, the determination of what consti- 
tutes validity in terms of a mathematical model is very 
straightforward: model validation is a systematic proce- 
dure for testing and modifying a simulation mathematical 
model to achieve the required level of fidelity in matching 
experimental data. So as a starting point in determining 
validation of an entire simulation complex, it makes sense 
to at least make sure the mathematical model on a stand- 
alone basis can be validated and then to go on and use the 
more subjective criteria he recommended for validation of 
the entire complex. So I am defining validation strictly 
from a standpoint of making sure the mathematical model 
that drives these systems has acceptable fidelity. 

The steps in validating a mathematical model are as 
follows: 

1 . Establish acceptance criteria 

2. Conduct flight tests and collect data 

3. Conduct simulation tests and compare results 

4. Analyzes discrepancies that exceed acceptance 
limits 

5. Modify the mathematical model to reduce dis- 
crepancies so they are within acceptance criteria limits 


I will go through each of these in more detail. The first 
step — the previous speaker made this point as well — is to 
establish the acceptance criteria. And that is very critical. 

It drives everything else from there on down. Once you 
have determined what is important to the missions you are 
trying to accomplish, then you can establish criteria to val- 
idate the model against those missions and then you can 
perform the rest of these activities: to conduct the neces- 
sary flight tests, and collect the data as a basis of compari- 
son; to perform simulation tests in an appropriate fashion 
to run comparisons with the experimental data; to analyze 
any discrepancies between the simulation results and the 
flight-test results; and, when those discrepancies exceed 
the acceptance criteria limits, to modify the mathematical 
model to bring those discrepancies within acceptable lim- 
its. The latter is, of course, the most difficult task. 

Let’s start with acceptance criteria, the first part of 
the procedure (table 1). I am going to define validation in 
two different ways: functional validation and physical 
validation. To begin with, functional validation, or accep- 
tance criteria to determine functional fidelity, basically 
requires fidelity of pilot cues. What you are trying to do is 
to make sure that what the pilot sees is an accurate repre- 
sentation of the input/out relationships of the aircraft. You 
don’t care what is going on inside the mathematical 
model. It is a black box. All you are really interested in is 
that given the right input you are getting the right output. 
That is functional fidelity. And this, of course, is the pri- 
mary way in which current training simulators are evalu- 
ated, on a functional basis. 

The kinds of criteria that are used for functional vali- 
dation are based on the effect, not on the cause. The 
response is being validated, not what is producing the 
response. The three classic criteria are trim, stability, and 
dynamic response. Regarding trim, you usually character- 
ize the control settings required to trim the aircraft at dif- 
ferent flight conditions. Often, stability is not specifically 
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Table 1. Acceptance criteria: functional fidelity 


1. Requires fidelity of pilot cues 

2. Functional criteria (validate effect) 

Trim 

Stability 

Response 

3. Tuning factors: empirical coefficients 

4. Scope of validation: validation at system level 

5. Bandwidth of validation: limited to handling-qualities range 

6. Amplitude of validation: limited to linear range 


used as a criterion in the training industry. It is somewhat 
inherent in the response measurements, but stability char- 
acteristics could be prescribed either in the frequency 
domain or in the time domain. For the frequency domain, 
the phase or gain margins can be specified; for the time 
domain, the eigenvalues or eigenvectors can be specified. 
The dynamic response of the actual test vehicle can be 
compared both in the time and the frequency domains 
with similar responses for the simulation to determine 
whether the response is correct. This is often used in the 
training industry, at least in terms of time-domain 
responses. There is very little in the way of frequency- 
domain criteria that is being used right now for validation. 

As far as the training industry is concerned, one of 
the major problems with the current acceptance criteria 
that have been established is that there is no attempt to 
specify how the manufacturer can tune the mathematical 
model to meet the acceptance criteria. The manufacturer 
basically has carte blanche to do whatever he needs to in 
order to meet those acceptance criteria. And typically 
what happens is the manufacturer will add empirical coef- 
ficients at appropriate places in the simulation that make it 
very easy to tune in order to satisfy the acceptance 
criteria. 

I have seen a number of cases in which scale factors 
and biases have been added to aerodynamic forces and 
moments. It is nonphysical, but it accomplishes the job of 
satisfying the specific test criteria. The problem with this 
kind of manipulation is that because it is done totally 
empirically, it satisfies the criteria at the test points but 
there is no guarantee that it is going to give a realistic 
response outside the test points or between test points. 
Basically, the test curve that the government gives you to 
satisfy is being fitted, and you can’t be assured that it is 
going to really represent the correct aircraft response. The 
other issues associated with the acceptance criteria are the 


scope of the validation. By this I mean is it end-to-end 
validation of the total aircraft that is of concern or is it the 
subsystems and their independent validation. 

Typically, right now validation is performed at the 
system level only, and it is based strictly on the six- 
degree-of-freedom aircraft rigid-body motion. If that is 
accomplished, the basic idea is that that is what the pilot 
sees, that is what the pilot perceives; there is no reason to 
carry validation into any more depth than that. The prob- 
lem with that, as we will shall see, is that it allows the 
manufacturer to tweak a subsystem, the tweaking of 
which may be totally inappropriate, in order to get the 
total response correct. If the rotor model isn’t right, he 
may alter the control system in order to give the net 
response that is desired. By allowing validation at the 
global level, the manufacturer is given a lot of leeway in 
adjusting individual components, which in turn eliminates 
interchangeability and modularity of the resulting 
simulation. 

Another major issue is the bandwidth of the valida- 
tion, that is, the frequency to which the simulation must 
be accurate (table 1). And typically there have not been a 
lot of frequency-response criteria associated with training 
simulators. This is a major problem. The way in which it 
is evaluated, though, does predominantly limit the band- 
width to the handling-qualities range, which again 
assumes that that is all the pilot is going to see and all he 
cares about. 

The last acceptance criterion, which is a really impor- 
tant issue, is the amplitude of the validation (table 1). 
Typically, people will limit the perturbations in the linear 
range. Validating the model when it is driven into its non- 
linear range is a much more difficult job. There are virtu- 
ally no acceptance test criteria that enforce driving the 
model into the nonlinear range to see if it is accurately 
represented. What you end up with is a training simulator 
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that has been validated only in the middle of the envelope 
for mild maneuvering. If a pilot maneuvers it aggressively 
or flies to the edge of the envelope, the simulation and 
that environment based on these validation criteria have 
not been validated. And that is precisely where simulation 
should be particularly valuable, in conditions in which a 
pilot would not want to fly a real aircraft. That is typically 
not addressed in the validation criteria. 

Let’s take the other alternative, which is physical 
fidelity (table 2). By this I mean we are requiring that the 
mathematical representations of the physical phenomena 
in the simulation be correct. Instead of looking at the sim- 
ulation as a black box where all you are interested in is 
proper end-to-end response, you are going to look at the 
way in which the phenomena are modeled and try to vali- 
date it to that level. This is typically done in engineering 
simulators. The main reason it has not been used in train- 
ing simulators is because it is a much more difficult pro- 
cess, much more costly to do and to validate, and, ulti- 
mately, because it is very difficult to perform in real time, 
which is required for training in real-time simulations. 
What is happening right now, however, is that with the 
advent of parallel processing technology and modern 
high-speed computers, we can take physically based 
models and perform real-time simulation with them. 

Computer technology has been developed to the point 
where we can start using physically based models for real- 
time training applications. As a result, we need to look at 
what the advantages are of this kind of modeling to the 
training industry. Again, the acceptance criteria in a phys- 
ically based model are to validate the cause rather than the 
effect. Here what you are going to try is to compare 
applied loads for accelerations of the vehicle for given 
flight conditions. The way in which the model is tuned is 
much more restrictive than it is in a functional model. The 
only way the contractor is allowed to modify the system 
is to modify the structure of the mathematical model, in a 


physically meaningful manner, or to change physically 
meaningful parameters, not empirical coefficients. So it 
tremendously complicates the process of tuning the simu- 
lation to match the acceptance criteria. 

The scope of the validation is another important 
issue. Now we are talking about validating the system at 
the subsystem level. It is not acceptable to think of this as 
just a black box — that as long as the right response is 
obtained, we don’t care what goes on inside. You are now 
going to break the total model down to a main-rotor mod- 
ule, a tail-rotor module, horizontal stabilizers, and 
engines. Each of the components is going to be separately 
validated against independent test criteria so the control 
system can no longer be altered to make up for problems 
in the rotor model. The bandwidth of the validation now 
has to be significantly increased. And it has to be 
expanded to include the bandwidth of all modeled degrees 
of freedom in the system. If the subsystems are going to 
be validated with physically based models, it is necessary 
that the degrees of freedom of all the physically based 
models in the system be exercised. Of course, it is neces- 
sary to be able to excite it throughout the range, to be able 
to go into the nonlinear region and validate it there. 

One of the benefits of going to physically based mod- 
els is that it should make it possible to achieve global 
fidelity of the mathematical model; that is, you should be 
able to drive it to the edge of the envelope, fly it with 
aggressive maneuvering, and really use it as it should be 
used, as a tool for training a pilot in dangerous flying 
activities, those he could never achieve or even come 
close to, safely, in an aircraft. 

The third item on the list was flight test and data 
acquisition (table 3). These have to be geared to the 
acceptance criteria. Once the acceptance criteria are estab- 
lished, data must be collected to support the performance 
of this acceptance test. What is done is to collect data 
associated with functional validation, trim data, stability 


Table 2. Acceptance criteria: physical fidelity 


1 . Requires fidelity of mathematical representation of physical phenomena 

2. Physical criteria (validate cause): applied loads/acceleration 

3. Tuning factors 

Model structure 

Physically meaningful parameters 

4. Scope of validation: validation at subsystem level 

5. Bandwidth of validation: includes bandwidth of all modeled degrees of freedom 

6. Amplitude of validation: excites nonlinear range 
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data, response data; typically this is limited to the airframe 
rigid-body motion. 

Physical validation is a much more difficult problem. 
In order to isolate subsystems for independent validation, 
it must be possible to collect boundary data at each of the 
subsystems. For example, the reaction loads between the 
rotor and the fuselage must be measured so that the rotor 
can be isolated from the fuselage motion and validated as 
an independent subsystem. Typically, therefore, it must be 
possible to collect load data at the subsystem interface and 
to be able to collect acceleration rate and displacement 
data at subsystems. As a result, it is a much more difficult 
data-collection task. 

The way in which this is commonly performed, or 
can be performed, is to use redundant sensors and kine- 
matic constraints to eliminate the instrument, calibration, 
and procedure errors that are encountered. Too often raw 
test data with no cross-checking are used for acceptance 
test criteria. Our experience has been that such data are 
fraught with calibration errors and procedure errors. There 
are too many good ways available for doing consistency 


testing, kinematic cross-testing, for this to be the case. 

This should be used to ensure that you have the right 
experimental data to form the basis of the acceptance 
criteria. 

The mass properties and the sensor geometry must be 
documented. It must be possible to perform maneuvers 
that span the bandwidth and amplitude of the validation 
criteria. For the closed-loop simulation, here for the simu- 
lation tests, there are two approaches. The purpose of the 
closed-loop simulation is basically to initialize the simula- 
tion to the starting test conditions, drive it with test con- 
trol inputs, and then compare its response with the 
dynamic response of the test (table 4). 

This is the way in which it is ordinarily done. The 
advantage is that it is simple to implement and requires 
minimal sensor data. The disadvantage is that you have a 
cumulative buildup of error and you cannot isolate subsys- 
tems because of the coupling between them. The open- 
loop approach to testing the simulation is to disable the 
airframe rigid-body motion and drive the simulation with 
the control inputs and the rigid-body motion that has been 


Table 3. Flight test and data collection 


1. Functional validation 

Collect trim, stability, and response data for airframe rigid body degrees of freedom 

2. Physical validation 

Collect loads data at subsystem interfaces and acceleration, rate, and displacement data at subsystems 

3. Perform data consistency tests with redundant sensors and kinematic constraints to eliminate instrument 
calibration errors and procedural errors 

4. Document mass properties, sensor geometry, and atmospheric conditions during tests 

5. Perform maneuvers that span the bandwidth and amplitude of the validation criteria 


Table 4. Conduct simulation tests and compare results 


Closed-loop simulation 

1 . Method 

Initialize simulation to starting test condition 
Drive simulation with test control inputs 

Compare dynamic response of simulation to dynamic response of test 

2. Advantages 

Simple to implement 
Requires minimal sensor data 

3. Disadvantages 

Cumulative error build up due to closed-loop integration limits validity of comparison 
^^__Cou£lin£be^eend^namicjubs^stemyjmtsjbilit^t^solm^iscre£ancies_^_ = _ 
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determined from the test data (table 5). So what you are 
really doing now is driving the simulation on in a dynamic 
wind-tunnel mode and looking at the loads that are pro- 
duced along the same flight trajectory that the aircraft 
produced. You compare these loads with those obtained 
from the flight to validate the model. The advantage is 
that it eliminates cumulative error buildup and it allows 
the subsystems to be validated independently. The disad- 
vantage is that it is much more difficult to implement, and 
more expensive data are required to isolate the loads at the 
subsystems. 

For the analysis and modification methods there are 
two primary objectives: model structure has to be estab- 
lished and the parameters have to be modified (table 6). 
And the kinds of modifications you will typically have to 
make are to add coupling, higher-order dynamics, and 
nonlinearities. 

The parameter identification method used for linear- 
parameter dependency can be regression. The more diffi- 
cult problem of nonlinear dependencies would require an 


output-error approach. The point I have been making all 
along is that training simulators are functionally validated. 
The validation is performed at the system level with the 
rigid-body airframe response as the validation criterion. 
Satisfaction of this criterion is achieved by tuning empiri- 
cal coefficients. The result is a model tuned for specific 
conditions that has been validated only for bandwidth 
low-amplitude maneuvers (table 7). 

The bottom line is that validation requirements drive 
the modeling sophistication (table 8). You get what you 
ask for. And the simulation manufacturers will not pro- 
duce the physically based simulation if the validation 
requirements are functionally based. For example, rotor- 
map models are functional approximations to the blade- 
elements model; they satisfy acceptance test criteria as 
currently specified. However, you could specify criteria in 
a form such that contractors would have to go to a blade- 
elements model in order to achieve your requirements. In 
conclusion, what I think is really needed is a standard for 
rotorcraft validation that in a sense is like the standard that 


Table 5. Conduct simulation tests and compare results 


Open-Loop simulation 

1 . Method 

Disable integration of airframe rigid-body motion in simulation 

Drive simulation with control inputs and rigid-body motion from test data 

Compare loads/accelerations of simulation with test data 

2. Advantages 

Eliminates cumulative error build up due to integration of airframe states 
Allows subsystems to be isolated and validated independently 

3. Disadvantages 

Implementation of simulation run is more difficult 
More extensive test data are required to isolated loads at subsystems 


Table 6. Analysis/modification methods 


Model structure determination 


1. Correlate errors to states and controls for nominal parameter values 

Statistical correlation of error 
Frequency response of error 

2. Postulate modification to model structure 

Additional coupling 
Higher-order dynamics 
Nonlinearities 

3. Repeat comparison step and iterate until error can be sufficiently limited by reasonable parameter changes 
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Table 7. Problems with current validation approach 


1 . Validation is only guaranteed in vicinity of test points 

2. Low-bandwidth validation does not support aggressive maneuvering, high-speed flight, or high-gain controllers 

3. Low-amplitude (linear) validation does not support aggressive or edge-of-the-envelope maneuvers 

4. Lack of subsysterrrvalklation elimmatesjnodu^ models 


Table 8. Validation requirements drive modeling sophistication 


You get what you ask for 

Simulation manufacturers will not produce physically based simulations if the validation requirements are functional 
Example: 

Rotormap models are functional approximations to the physically based blade-element model 

They satisfy trim and stability requirements and low-bandwidth response requirements for function validation 
They will not satisfy a validation criteria that specifies rotor impedance (rotor load frequency response to hub 
acceleration) 


Table 9. Rotorcraft validation standard 


A standard for rotorcraft validation is required that will address the following: 

1 . Acceptance criteria versus simulator mission requirements 

2. Flight-test procedures and instrumentation versus acceptance test criteria 

3. Generation of simulation data and comparison with flight data 

4. Model structure determination and parameter identification methods for reducing errors to specified limits 

5. Acceptable physically based parameters for tuning and their allowgblgjange^fvariat^ 


we are addressing here this week for simulation qualifica- 
tions (table 9). It could be either a part of the simulation 
qualifications or be detailed enough to require a separate 
specification. 

We have to define the acceptance criteria as a func- 
tion of the mission requirements. We have to determine 
flight-test procedures and instrumentations in order to be 
able to implement acceptance criteria. We have to be able 
to generate the simulation data and compare them with 
flight data in a systematic manner, apply modern tools for 
model structure determination, and parameter identifica- 
tion for achieving the criteria. Then we have to determine 
what physically based parameters are acceptable for tun- 
ing the simulation and what is their allowable range of 
validation. These are all terms that should be defined in a 
specification so that validation can be standardized. 

MR. WALKER: Since the interface between the sub- 
jective evaluator and the mathematical models is really 
the simulator that is provided by visual systems, motion 


bases, audio systems, and so on, how do you resolve the 
errors that may be introduced by these systems in the 
development of your validation? 

MR. DU VAL: I am referring strictly to the validation 
of the mathematical model; my contention is that you 
should not compensate for errors in these other systems 
by modifying the mathematical model; you should put in 
compensations for the systems, where they belong, that is, 
within the systems. 

MR. HAMPSON: I agree entirely with you. I do have 
some difficulty, though, with some of the comments you 
made with respect to tweaking the model. I don’t know if 
this is particularly a helicopter problem you are address- 
ing, but certainly with fixed-wing and also with the heli- 
copter models that are provided by the aircraft manufac- 
turer, we, as a simulator manufacturer, do not tweak the 
models. We identify the deficiencies and go back to the 
aircraft manufacturer and tell him there is something 
wrong with his model or have him explain to us why we 
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have a problem. And I think that is the proper way to do 
things, rather than expecting the simulator manufacturer 
to tweak a model. 

It goes back to something I said yesterday, but in the 
helicopter world we rarely get a model from the manufac- 
turer of the aircraft. That is a significant issue, I think. 

MR. DU VAL: That is true. I haven’t really made the 
distinction of whether the mathematical model was gener- 
ated by the simulator manufacturer or the aircraft manu- 
facturer. The point is if the physically based mathematical 
model does not match the acceptance criteria, to add 
empirical parameters to make it match the criteria is not 
an appropriate solution, that it must be physically 
modified. 

MR. GALLOWAY : You mentioned that you get 
what you ask for. I would like to add the comment that 
you get what you pay for or are willing to pay for. How 


do I convince my Navy program managers to pay for the 
efforts you advocate for getting the data? 

Mr. DU VAL: The answer is modularity. You are 
going to pay for it in the short term, but you are going to 
get your money back in the long term. If you validate the 
subsystem models at the subsystem level, then you have 
interchangeability of mathematical models. You can plug 
in rotor models, you can build on them, because you vali- 
dated each of these components separately. It provides for 
the kind of modular interchangeable mathematical model- 
ing for simulation that we have been striving for. Once 
you have validated the basic component it is only a matter 
of changing the physical attributes to validate it with a dif- 
ferent aircraft. So even though it is more costly to do this 
up front, it is going to reduce the cost of validation on 
future simulation activities because you have building 
blocks you can work from. 



Ronald W. Du Val is president of Advanced Rotorcraft Technology, where he has 
developed a team with a reputation for excellence in the fields of rotorcraft simula- 
tion and analysis. He received a B.S. in mechanical engineering from the University 
of California, Berkeley, an M.S. in systems engineering from the University of 
Houston, and a Ph.D. in aerospace engineering from Stanford University. Dr. Du Val 
worked for NASA at the Johnson Space Center, where he assisted in the develop- 
ment of simulations in support of the Apollo missions, and where he participated in 
the initial design of the space shuttle’s reentry and terminal-area guidance and con- 
trol systems. He later transferred to NASA Ames Research Center where he applied 
methods of state space control to the problems of rotorcraft. Dr. Du Val left NASA in 
1982 to set up Advanced Rotorcraft Technology. 
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15. FREQUENCY-RESPONSE TECHNIQUES FOR DOCUMENTATION 
AND IMPROVEMENT OF ROTORCRAFT SIMULATORS 


MARK B. TISCHLER 


I would like to pick up on a number of points that 
Ron Du Val made. It was a good introduction for some 
more of the detailed aspects, and I think it follows well 
with what Dave Key is going to talk about afterward. I am 
going to talk specifically about analytical techniques, 
some of which Ron introduced for documenting and 
improving rotorcraft simulation. This includes mathemati- 
cal modeling, which Ron was addressing, and visual and 
motion systems, how we do that documentation, and how 
we tweak the model, as was discussed. 

I would like to cover the background of the general 
topic, which is system identification, a class of techniques 
for documenting both the mathematical model and the 
implementation in the simulator. The specific approach 
that I have been working on and what we use at Ames 
extensively is the frequency-response approach. It is an 
input/output validation technique, but can be used to doc- 
ument and to validate physical models. Specifically, we 
are going to look at the application of system identifica- 
tion to a variety of validation problems. The core of my 
presentation is going to be a series of illustrations of how 
we used the technique for a number of simulators, includ- 
ing the UH-60, AH-64, and STOVL. 

I will show you a potpourri of illustrations, how these 
techniques are used, how you interpret them, and finish 
off with a summary. 

As I mentioned, the overall class of techniques is 
included in the category of system identification. And for 
those who are not familiar with system identification, it is 
a procedure by which a mathematical description of an 
aircraft, in this case a rotorcraft, is extracted from flight- 
test data. In this respect it is the inverse of simulation. In 
simulation we make such assumptions about the charac- 
teristics of the aircraft, its aerodynamics, how many 
degrees of freedom it has, etc., and based on those 
assumptions we formulate a physical model, and generate 
a simulation that is intended to predict aircraft motion. 
When all that works and the predicted aircraft motion 


equals measured aircraft motion, we have a good 
simulation. 

Unfortunately, as has been pointed out a number of 
times, that is often not the case. It is very difficult to fig- 
ure out how to change the mathematical model on this end 
to update the simulation and make these two things match. 
One of the most sophisticated ways of making that happen 
is to work the problem in reverse. That is, take aircraft 
data, go out and do special flight tests for system identifi- 
cation; system identification becomes an inverse proce- 
dure by which one extracts a mathematical model from 
the flight tests. These can be physical models, transfer- 
function models, or state-space models. Once these mod- 
els are extracted they represent the exact characteristics of 
the aircraft. Then they can be compared back to back with 
the simulation, the simulation models can be updated, and 
a comprehensive method is produced, by which both the 
mathematical models and our physical understanding can 
be updated. We may want to go back and change some 
assumptions; maybe, for example, some of our mathemat- 
ical assumptions were not good. 

Typical examples of the uses of system identification 
are given in figure 1. System identification has been 
around a long time, but only recently has it been adopted 
in a broad way in the rotorcraft community — in the last 5 
to 10 years. The reason is, there are special problems 
associated with it that make it more difficult in some ways 
than a standard fixed-wing problem. 

In rotorcraft there is a high-level rotor noise. The 
helicopter is inherently a very high-order system, so the 
system cannot be decoupled, unlike in fixed-wing work 
where only a small subset of transfers is identified. Gener- 
ally, instead of having to identify 1 0 or 20, as many as 40 
or 50 might have to be identified. There is a great degree 
of high-axis coupling. You have to go at least six- or 
nine-degrees-of-freedom, and helicopters are generally 
unstable machines. I am not going to go through in detail 
the engineering aspects of system identification (shown in 


131 



• What is rotorcraft system identification? 


Frequency-Response Techniques 



132 


Figure 1 . System identification background. 
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fig. 2). There are a lot of papers about it, papers by Ron, 
me, and others in the audience here. Frequency-sweep 
testing of the aircraft is conducted to generate a data base. 
Then, data compatibility is used to make sure the data are 
good, state-estimation is used to reconstruct poorly mea- 
sured states, and advanced FFTs are used to convert the 
time-domain data to frequency-response data. 

The frequency response is a complete description of 
the aircraft. It is a linearized description, but it is a lin- 
earized function of a nonlinear function. In that respect it 
does fully characterize the aircraft. For a lot of what we 
want to do, this is sufficient, because we can characterize 
the aircraft behavior by its frequency response and com- 
pare that with the simulation frequency response. I am 
going to show you an illustration of that. 

In handling qualities we work with frequency 
responses of the system to check bandwidth. You can use 
advanced techniques for extracting from the frequency- 
response stability-control derivative models. This is 
important. I will show you an example in which we used 
such a model and actually flew it in a piloted simulation. 

In a number of simulations we implement a stability and 
control derivative look-up table as a function of flight 
condition. This is one way of actually generating a simula- 
tion model for piloted simulation. Finally, we want to ver- 
ify that these identified models are correct by checking in 
the time-domain. 

This is sort of the overall road map and I will not go 
into any more detail. Let me just point out a couple of rea- 
sons why we like the frequency-response approach for 
rotorcraft. First, the frequency-response technique has the 
advantage in that when you form the frequency-response 
ratio, the uncorrelated effects of process and measure- 
ments noise drop out. That is, any noise source that is not 
correlated to the input drops out of the calculation. And 
that makes identification easier. You do not have to make 
an assumption about the noise or you don’t have to iden- 
tify it. So from a technical standpoint it has some advan- 
tages, especially for a helicopter in which the data are 
often quite highly contaminated by noise, by turbines, or 
by measurement noise. 

Second, you can extract parametric models in the fre- 
quency range where the data are valid. We have access to 
the function called the coherence function, which gives 
you direct measurements of the accuracy of the data. If 
the coherence drops in a particular frequency range, you 
may go out and rerun the data and go for it again. 

Third, you can estimate time-delays directly, because 
the phase shift is a linear function of time-delay. It is very 


important in simulators where you want to identify time- 
delay. Then there is integration in the time and frequency 
domains. There are methods for artificially stabilizing the 
system; they do not work very well for highly unstable 
rotorcraft. Frequency domain does work well for that. All 
the results I will show you are for unstable systems. 

Finally, we have developed a comprehensive package 
for the frequency-domain approach, CIFFR, for Compre- 
hensive Identification From Frequency Responses. Appli- 
cation of system identification to the simulation environ- 
ment in sort of a broad sense is depicted in figure 3. The 
pilot is going to make inputs into a mathematical model, 
which produces estimates of what the aircraft is doing. 
That may drive the visual system through its compensa- 
tion, and the motion system through wash-outs and 
motion drives. The pilot is subjected to these cues, and 
they may be matched or mismatched and produce an 
overall percipient. The frequency approach that I’m going 
to talk about is applicable to all aspects of the validation 
process. 

You can calculate frequency responses between pilot 
inputs and aircraft states and validate the mathematical 
model alone. You can look at aircraft states, to the visual 
system, and characterize the motion-system response, or 
go end-to-end and characterize the overall response. One 
example has been mentioned, the XV- 15. We suppressed 
the actuator dynamics, because those delays were com- 
pensated by the visual systems dynamics, and because we 
knew that there were going to be extra delays in the visual 
system and that the end-to-end response would be okay. 
That is an example of where you might shift some of the 
delays and get the same end response. Some examples of 
what we have done in the past (and there are papers on all 
of these) are what I am going to highlight in the remainder 
of my discussion. I mentioned the XV- 15; it was highly 
validated both in the time and frequency domains, and 
was a very good example. 

I think most people involved in the XV- 15 would 
agree that it was probably one of the best simulations ever 
run at Ames. The transfer of training was excellent, and 
most of the papers by Ron Gerdes and Dan Dugan indi- 
cate that the pilots were amazed when they got into the 
aircraft. The frequency-response studies that were done 
indicated that the validation was excellent across the 
whole pilot-handling-qualities range. We have done quite 
a few studies over the years on the UH-60. 1 will talk 
about some work on STOVL simulation. There has been 
considerable effort recently in characterizing the VMS 
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Figure 2. Frequency-response method for system identification. 
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Figure 3. Application of frequency-response procedures to simulation/simulator validation. 
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motion base and visual systems, and I will present some 
results from that. 

Frequency-response testing was used heavily in vali- 
dating the LH simulation, both in terms of characterizing 
its response and of validating the handling qualities. The 
Army Test Directorate (AQTD) had our software in a 
portable suitcase and actually characterized the frequency 
responses in the lab. And then we have recently made, as I 
mentioned, an Apache mathematical model extracted 
from flight-test data. 

The Blackhawk study that was reported by Mark 
Ballin at the last AHS meeting is shown in figure 4. We 
did frequency sweeps; here is an example. The pilot gen- 
erates the inputs; we are not in favor of computer- 


generated input. The pilot supplies a good input. In this 
case we are interested in validating the simulation mathe- 
matical model. It is a physical based mathematical 
model — it is a blade-element-type model, very sophisti- 
cated. This is our input into the system. We use 
frequency-response techniques to identify input to output 
frequency response of the model itself, and of the aircraft. 
Figure 4 shows the pilot’s input to the aircraft. 

In figure 5, the solid lines are magnitude, phase, and 
the coherence function. When the coherence function is 
high, it indicates the data are accurate. In this case they 
are accurate, and include the rotor dynamics. In fact, the 
notch shown in the coherence-function curve is an effect 
of the lead-lag motion of the blades. 




Figure 4. UH-60 Black Hawk Frequency-Sweep Flight Tests in Hover (from Ballin, 1990 AHS forum). 
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Frequency (rad/sec) 


Figure 5. Tuning Howlett inflow model for improved roll correlation. 


You can see that the baseline model, the dashed lines 
in figure 5, is pretty good at high frequency. The rotor 
dynamics are pretty well approximated and things look 
good beyond 1 rad/sec. Below that, there is quite a bit of 
error between the baseline model; it turns out the prob- 
lems were associated with inflow dynamics. There is a 
first-order inflow model, referred to as the Howlett model. 
When the model was developed there was no way of tun- 
ing the coefficient; there were no flight-test data at that 
time, and this provided opportunity to collect some. By 
adjusting a couple of the aerodynamic constants in the 
inflow equations we were able to bring the model into 
very close agreement with the fight-test data; this 
response is very close to the more sophisticated, so-called 
Pitt Peters model; it is an example of how this tuning. 


which was discussed before, is done. You can get a very 
detailed characteristic of how the model changes by tun- 
ing the aerodynamic parameters. In this case the pilots 
reported a great improvement in their perceptual opinion 
of the characteristics of the simulator. 

The next program I want to talk about is the Apache. 
We ran a series of frequency steps, in late August 1990. It 
was a very comprehensive program, with a variety of 
goals, one of which was to validate the AH-64 mathemat- 
ical model. We have a couple of mathematical models 
from one of the manufacturers and one of them was 
developed in house. We did frequency sweeps in hover 
and in forward flight with the SAS off, and gathered quite 
a data base from that. One of the goals of the program was 
to extract a linear model, which was then used in the 


137 





Frequency-Response Techniques 


simulator to do handling qualities. In this case, the study 
was done to evaluate the displacement dynamics and to 
determine how they affect pilot handling qualities. The 
point is, we extracted a mathematical model and actually 
flew it. That was one of the first times that had been done. 
The hover response, SAS off, is shown in figure 6; the 
figure also shows the on-axis pitch response, the on-axis 
roll response, magnitude, and phase. The flight data are 
represented by a solid line, the dashed line is the model. 

We identify a model that, as you can see (fig. 7a), 
characterizes response very well. This particular model 
has basic rigid bodies of freedom. It also has in it the 
inflow degree of freedom, and you can see that the charac- 
terization is quite good. In the time-domain (fig. 7b), the 
model is very characteristic of the on-axis response to 
pedal input, yaw rate, and acceleration, which is very 
good. And the dominant coupling response, which is a roll 
response, is excellent. So this is an example of where we 
took the model and drove it with these similar flight-test 
data; as you can see, the predictions are really excellent. 
The pilots reported very good fidelity (fig. 7b) of the sim- 
ulation, that the coupling responses are very good, and 
that they are actually flying this model. 

Another example is the STOVL program (fig. 8). In 
this case we wanted to extract a linear model. You have 
the possibility of generating a linear model, but you can 
also use system-identification techniques to do the same 
thing. And when you use system-identification techniques 
to do that, you can characterize some of the nonlinear 
behaviors much better. 

The step input into the elevator, which is the domi- 
nant longitudinal response, is shown in figure 9. The 
dashed line is the numerical perturbation model. In fact, 
for the very beginning of the response the numerical per- 
turbation technique is much better because it is a very 
small perturbation. And as you can see, it is unstable. 

Our last example is a vertical motion simulator, 
which is a lead-in to Dave Key’s presentation. Here we 
were interested in documenting the vertical motion simu- 
lator response, both the visual system and the model 
response, as well as the motion system (fig. 10). The 
model response — and it is an ideal, simple model — is the 
solid line; it is a very simple attitude system. Our visual 
system drive uses an algorithm developed by McFarland 
to buck out the inherent delay, and the resulting response 
is exactly on top of the model (fig. 10). He did a very nice 
job in coming up with an algorithm that allows the system 
to follow the mathematical model. 


The motion command has a great deal of wash-out at 
low frequency, and tracks with some gain error at high 
frequency the motion follow-up which the pilot feels, lag 
at high frequency (fig. 10). The system-identification 
approach provides a way to characterize independently all 
these various effects; Dave Key will talk about how you 
interpret that. The point is, you go into the simulator and 
split out the various effects. You can see that at low fre- 
quency the motion wash-out is quite significant. The last 
result (fig. 1 1) shows a comparison of pilot workload in 
the UH-60 in a hover/bob-up task. Here we are looking at 
the frequency contents, and what I have plotted is fre- 
quency range versus the rms of the pilot stick input over 
the total rms. 

What figure 1 1 shows is that most of the pilot’s 
input — say up to about 80% of it, which is reflective of 
the crossover frequency— is at 2.5 rad/sec. That indicates 
the pilot is operating at a crossover frequency of 
2.5 rad/sec. The flight data are indicated by the open cir- 
cles; you can see the characteristics are almost on top of 
each other. In fact, the pilot ratings are essentially the 
same. I think they were off by one pilot rating. It is 
another way of using the frequency-response method to 
calibrate workload and to get transfer-of-training issues, 
because a pilot from 1 to 1 0 rad/sec is operating the same. 

Summarizing, I think you can see that system- 
identification techniques are comprehensive and allow 
you to look at the whole range of problems. They are very 
well suited to rotorcraft and provide a great deal of physi- 
cal insight. Finally, there are a number of computational 
tools out there for doing this analysis; Mathematical Lab, 
Control C, and CIFFR. 

Are there any any questions? 

MR. BRICZINSKI: I think that your implication of 
using this frequency-response technique primarily can be 
used to complement, to help analyze, simulation models 
as opposed to generating them. I think your techniques of 
system identification will generate a linear small- 
perturbation model. We find it necessary in your field to 
use a full force and moment type model. Did you suggest 
perhaps generating maps of stability derivative-type 
models that could be interpolated and then serve in a sim- 
ulation technique? 

DR. TISCHLER: Some of the best simulation models 
of helicopters in fact have been done by easily program- 
ming table look-ups at every 20 knots of perturbation 
derivatives. You can put in the aerodynamics and then the 
gravity and kinematics in a nonlinear way. 


138 


I 













PHI / 0.18E-01* ROLLATT P / 0 IBE-OI* ROLLRATE 


Frequency-Response Techniques 



141 


Figure 7. Concluded. 


Identification of bare-airframe responses to 













Figure 9. Comparison of time-domain predictions: numerical perturbation versus identification. 
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Figure 10. Documentation of vms: motion and visual systems (Atencio). 
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Figure 1 1. Comparison of pilot workload: UH-60 bob-up task (Atencio). 
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Of course if you are going to try to get the edges of 
the envelope, you are not going to make it. If you are talk- 
ing about in and around the reference points, they are 
quite accurate. In fact the frequency sweeps, if you look at 
some of the papers, show pretty extreme responses. The 
aircraft was at the edges of its envelope and yet the linear 
approximations were pretty good. 

MR. BRICZINSKI: We are progressing in the rotor 
modeling from quasi-map methods to a more rigorous 
blade-element method to say we are going to go where it 
might go for coefficient map models and take our entire 
aircraft as opposed to the rotor and go to quasified map- 
ping models. 

DR. TISCHLER: I am suggesting that there are some 
applications, for example, in this Apache case, in which 
we were interested in looking at the hover characteristics. 
We have no outside visual cues so that you are not going 
to be maneuvering off the edges of the envelope. You are 
flying on one eye and operating your hover. Clearly, it is 
appropriate there. The computers are now such that you 
can run these very sophisticated mathematical models 
without always making those approximations. What I am 
saying is it provides a mechanism for validating those, 
and there may be some situations in which that sort of 
characterization is enough. I would not say that that is 
generally true. Just as an example, an illustration of how 
you would use it. 

MR. McFADDEN: My question is, do you find that 
small discontinuities in nonlinearities at neutral are a 
problem, or can you ignore them? 

DR. TISCHLER: It depends on what kind they are. 
We did a characterization, for example, of the ADOCs 
system and it has nonlinear stick sensitivity, which is very 
common. If you have a small dead band and if you are 


operating through the dead band, that has a linear describ- 
ing function. But it has a phase effect and that’s a mess. 

So it depends on how severe they are. If they are simple 
nonlinearities they can be accurate. 

MR. CARDULLO: There have been considerable 
attempts to use parameter identification techniques to 
identify full force and moment nonlinear models for 
fixed-wing airplanes and they have been quite successful. 
Do you have any plans to try to develop this technique for 
rotary-wing nonlinear models? 

DR. TISCHLER: I think there is some work going on 
in that field. I think Ron Du Val has worked to some 
extent in that field. It is a very tough one because the 
parameters that you are talking about in a full-force model 
combine in a very nonlinear way and in a highly corre- 
lated way. If you look at the sensitivity of some of these 
parameters, there isn't any. In terms of the input/output 
characteristics, you need a lot of detailed inflow in the 
component sense. You need accurate measurements. The 
problem with rotorcraft is that the measurements have not 
been made. If you look, for example, at longitudinal 
response, how are you going to do a correlation based on 
validating the X-force when there isn’t any in a 
helicopter? 

MR. CARDULLO: SCT has been doing some work 
with the V-22, 1 think. 

DR. TISCHLER: Yes. And they have done a lot of 
work on the Harrier. They have encountered a high level 
of correlation. If you start introducing a lot of effects, they 
found things dependent on squares and cubes of whole 
inputs; everything was correlated. It is difficult. You need 
measurements of the individual components. It can be 
done but it is difficult. 



Mark B. Tischler is rotorcraft group leader for the Army Aeroflightdynamics Direc- 
torate at NASA Ames Research Center, in which position he is in charge of research 
activities in rotorcraft system identification and high-bandwidth flight control. 

Before joining the Aeroflightdynamics Directorate in 1983, Dr. Tischler worked at 
Systems Technology, Inc., where he conducted analyses of a variety of fixed- and 
rotary-wing vehicles. He received B.S. and M.S. degrees in aerospace engineering 
from the University of Maryland, and a Ph.D. from Stanford University’s Depart- 
ment of Aeronautics and Astronautics. Dr. Tischler is a U.S. member of the AGARD 
Flight Mechanics Panel. 


146 


N 93-30690 




u 




16. BANDWIDTH AND SIMDUCE AS SIMULATOR FIDELITY CRITERIA 

DAVID KEY 


Many characteristics define a visual system’s quality: 
the field of view, the resolution, the detail, and, what I 
will talk about, the delays in response. In addition, I will 
tallc about how to make an overview of the total visual 
cuing quality. 

Bandwidth has been mentioned several times today. I 
will define it in the context of handling qualities. I will 
show how the visual delays affect the bandwidth and the 
handling qualities, and how we could use that to assess 
the simulation fidelity. The first paper this morning raised 
many questions about how much fidelity you need for 
transfer of training. The report the author referred to then 
(ref. 1) was one I worked on back in 1 980. We asked the 
same questions 1 1 years ago. My field is handling quali- 
ties, not training, so I still do not have the answers. But I 
will give some hint of how I think you can interpret 
fidelity. 

Figure 1 shows a page out of the handling-quality 
specification ADS-33 (ref. 2), and defines bandwidth. For 
a rate-response type, the bandwidth is the lower of the 
gain margin or the phase margin. For an attitude- 
command/attitude-hold system, you use the phase margin. 

Figure 2 shows the bandwidth boundaries in the han- 
dling quality specification. Target acquisition and tracking 
requirements are not appropriate for many civil aircraft. 
More appropriate would be the boundaries for “all other 
MTEs in Usable Cue Environment UCE = 1 .” UCE is 
defined in reference 2. Essentially, a UCE greater than 1 
implies degraded visibility, and I will limit this discussion 
to the context of day visual requirements. 

Figure 3 shows the UH-60 Black Hawk helicopter’s 
frequency response, gain, and phase. If we put 100 msec 
of pure delay into the system, it does not affect the gain, 
but it does affect the phase. Reading the bandwidth (it 
turns out that the Black Hawk is gain-margin limited), the 
result can be plotted on the roll bandwidth requirement 
(fig. 4). With 100 msec of delay, the response moves 
much closer to the Level 2 boundary. Thus, with an extra 
100 msec of delay, the Black Hawk would have changed 


from a really good (Level 1) almost into the region of 
degraded handling qualities (Level 2). The levels of 
flying-qualities concept (ref. 2) is based on the Cooper- 
Harper Pilot Rating Scale (ref. 3). The Cooper-Harper 
pilot rating scale provides a measure of subjective evalua- 
tions of handling qualities. Ratings from 1 to 3.5 imply 
that the aircraft is good, has desirable performance, and an 
acceptable workload. At ratings between 3.5 and 6.5, the 
aircraft is not so good (Level 2). The pilot can still do the 
job, but with only adequate performance and the workload 
is increasing. Above 6.5, the aircraft is so bad that the 
pilot can no longer do the task, but should not lose control 
(Level 3). 

So, we can see that with an added 100-msec delay the 
Black Hawk response goes from very good to marginal, 
that is, almost into the Level 2 region. Now what does that 
mean in the simulation world? Figure 5 is a timing dia- 
gram for the VMS at Ames Research Center. Starting at 
the pilot’s controls, there are some delays or dynamics in 
the artificial feel system, then there are some measure- 
ment delays, then signals go into the main host computer, 
which has a 20-msec cycle time. Finally, the computed 
aircraft response comes out to drive the CGI and the 
motion base. Nominally, the CGI operates at 60 Hz and 
effectively takes 2.5 cycles, so it adds an 83-msec delay. 
The motion base can add an equivalent delay of 70 msec 
in pitch and roll and up to 160 msec in heave. The motion 
dynamics are not truly a pure delay, but can be repre- 
sented as such for the frequency range of interest (<3 Hz). 

When the pilot moves the control, he can only tell 
how the helicopter responds by the response of the visual 
and motion system. As far as he is concerned, this is the 
airplane. He cannot distinguish delays in the visual and 
motion cuing from delays in the mathematical model — 
that is, from the aircraft being simulated. This hypothesis 
sounds obvious, but we have performed an experiment to 
demonstrate the fact (ref. 4). The configurations tested are 
shown in figure 6. The fastest configuration had a roll 
damping Lp = 4. This would have a bandwidth = 4 with 
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Figure 1. Definition of bandwidth and phase delay. 
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Figure 2. Handling-qualities boundaries for pitch and roll (hover), (a) Target acquisition and tracking (pitch), (b) target 
acquisition and tracking (roll), (c) all other MTEs - UCE = 1 and fully attended operations (pitch), (d) all other MTEs - 
UCE = 1 and fully attended operations (roll), (e) all other MTEs - UCE > 1 and/or divided attention operations (pitch 
and roll). 
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Figure 4. Effect of delay on bandwidth and phase delay, (a) Target acquisition and tracking (pitch), (b) target acquisition 
and tracking (roll), (c) all other MTEs - UCE = 1 and fully attended operations (pitch), (d) all other MTEs - UCE = 1 and 
fully attended operations (roll), (e) all other MTEs - UCE > 1 and/or divided attention operations (pitch and roll). 
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Figure 6. Handling-qualities variations with visual delays, (a) All other MTEs - UCE = 1 and fully attended operations 
(pitch), (b) all other MTEs - UCE = 1 and fully attended operations (roll). Attitude command response types, (c) All 
other MTEs - UCE = 1 and fully attended operations (pitch), (d) all other MTEs - UCE = 1 and fully attended operations 
(roll). Rate response types. 


no delay. However, there were some delays from the the visual delay can be made zero. To investigate the 

computation times, so actually it has a bandwidth of effects of delay in the visual system compared with the 

about 2.8. mathematical model (aircraft response), the basic visual 

Dick McFarland of Ames has generated a scheme for delay was compensated or, alternatively, a delay was 

compensating for the CGI delay (ref. 5) in such a way that added further downstream as though it was part of the 
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mathematical model. Those two points lie on top of each 
other on the bandwidth plot (fig. 6). Similar combinations 
of delays up to 0.383 msec were investigated. The 
handling-qualities pilot rating was 3.2 (Level 1) with no 
delay, and with 0.383 delay the pilot rating was 8. So it is 
clear that the pilot ratings do indeed degrade as delays are 
increased, and the ratings correlate well with the 
ADS-33 bandwidth boundaries. Also, as hypothesized, 
the pilot cannot tell the difference between delays in the 
visual and delays in the mathematical model. 

When we consider motion cues, the situation is a bit 
more complicated. The helicopter model was a very sim- 
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pie first-order one. Figure 7 shows the Bode plot for the 
motion. If we add the stick dynamics, the phase and gain 
are changed as shown. But the motion cue not only has a 
delay, it has to have washout to limit excursions; this 
changes the response even more. Consider the cab 
response between 1 rad/sec and the bandwidth (5 rad/sec), 
the region that is really of interest. The gain is about 8 dB 
down (a factor of about 6). Roll would be down by a fac- 
tor of about 2. Phase matches the model exactly at about 
2 rad/sec. At 1 .0 rad/sec, there is about 45° of phase lead, 
and at 5 rad/sec there is about 45° of lag. 




Figure 7. Motion base frequency response (pitch angle response to stick force). 
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Figure 8 shows pilot ratings obtained with and with- 
out 83 msec of delay, with and without motion. The first 
point to make is that for each of these tasks motion is bet- 
ter than no motion. The next point is a question: How do 
you combine the visual and motion dynamics? Should the 
visual and motion be matched or should we try to com- 
pensate for the visual time delay? We do not have an an- 
swer to this, but do plan an experiment to investigate it 
later in 1991. In the meantime, it would seem reasonable 
to set the visual delay to match the fastest axis of motion. 

Back to the question of how much delay should be 
allowed in the visual system? My suggestion is to allow 
the stick-to-visual bandwidth to degrade to Level 2 
(figs. 3(c) and 3(d)), but do not go out of the Level 2 
region. Level 3 means the pilot cannot do the task. Pre- 
sumably, if the handling qualities are so bad a test pilot 
cannot fly the task, then it is unlikely to give a very good 
transfer of training. If the helicopter itself is Level 3, you 
can match the helicopter, but if you are training to fly a 
Level 3 helicopter, there are other problems that need 
fixing before routine training starts! These points are 
summarized in table 1 . Note that a fixed value of delay 
such as 100 ms may or may not cause these boundaries to 
be violated, depending on the bandwidth of the helicopter 
being simulated. 

Now consider the question of how to assess overall 
visual cue fidelity. In developing the handling quality 
specifications (ref. 2) we had to address flying qualities in 
a degraded visual environment, such as when flying at 
night with night-vision goggles. Many parameters such as 
field of view, resolution, scene detail, and response 
dynamics influence the cue fidelity so that it is currently 
impossible to compute a cue fidelity. As an alternative we 
invented a subjective scheme for evaluating how well the 
pilot could see and called it the usable cue environment 
(UCE). The procedure is essentially as follows: Take a 
helicopter with good Level 1 rate response in day visual 
conditions and assess its capabilities in the degraded 
visual environment. Thus, on an appropriate dark night 
with clouds, rain, etc., with the vision aids to be used, 
perform precisely defined tasks and ask the pilot to rate 
how precise and aggressive he can be. The process is 
summarized in figure 9. To get an assessment of the simu- 
lator visual cues, we can apply the same procedure 
(table 2). We call this SIMulator Day UCE; that is where 
“SIMDUCE” comes from. If the cues are as good as they 
would be during the daytime, SIMDUCE = 1 . If the 
SIMDUCE = 2 or 3, it is roughly equivalent to having 


Level 2 or Level 3 handling qualities, so the SIMDUCE 
number could be treated the same way as the degradation 
caused by delays. That is, SIMDUCE = 2 is probably sat- 
isfactory for training. If SIMDUCE = 3, it is not satisfac- 
tory. We applied this routine to the NASA VMS simulator 
and obtained the data shown in figure 10. This shows the 
average and standard deviations and an overall UCE of 3. 
The VMS visual is not inherently that bad; we were trying 
to get degraded UCE so had put in “fog.” For the FAA to 
incorporate the SIMDUCE concept into an advisory circu- 
lar, they will have to define a Level 1 rate-response type 
helicopter mathematical model. This should be a standard- 
ized model, and it could be made very simple — I do not 
expect manufacturers would mind too much. 

My conclusions can be summarized as follows: 

1. For simulator delays, the visual and motion 
delays should be set approximately equal. Then the 
bandwidth from the stick, all the way through to the visual 
response, should be no worse than ADS-33C Level 2. A 
single value of delay such as 100 msec will not achieve 
this and should not be used. 

2. Use the SIMDUCE procedure to get an overall 
calibration of the cue fidelity and it should be 1 or 2, 
not 3. 

Are there any questions? 

Questions 

MR. McFADDEN: I won’t leave you without a ques- 
tion, David. What frequency response was the VMS when 
you used it there? You showed a frequency response. Do 
you recall? 

MR. KEY: I am not sure I understand your question. 

MR. McFADDEN: Where was your 45° phase 
margin? 

MR. KEY: Okay. The 45° phase on the cab response 
to stick was around 2 rad/sec. 

MR. McFADDEN: Thank you. 

MR. KEY: That is not the response of the VMS to a 
pure input to the VMS motion. That response is through 
the washout. This is the way we had it set up with the 
washout. 

MR. McFADDEN: I understand. You could have 
made it better. 

DR. TISCHLER: Right. 

MR. GREEN: The question I have is how do you 
treat saturation or the limited throw relative to the cab? In 
other words, of the motion base. 
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Figure 8. Comparison of handling qualities with and without motion, (a) Hover, (b) vertical translation, (c) pirouette, 
(d) slalom, (e) bobup, (0 dash/quickstop, (g) sidestep. 
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Figure 10. VCR/UCE results. 
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Table 1. Application of bandwidth to simulation fidelity 


Criteria for simulator delay limits: 

How to combine visual and motion cues? 

Match visual and motion (rather than each as fast as possible) 

How much handling-qualities fidelity for transfer of training? 

Do not allow stick to visual BW worse than Level 2 (or match the helicopter) 


Table 2. SIMDUCE: calibration of visual cue fidelity 


Obtain VCR as for UCE except: 

Simulator, not flight 

Day, not degraded visual environment (DVE) 

Task performance standards for day, not DVE 
Rating is SIMulatorDayUCE (SIMDUCE) 

Should be 1 if cues are as good as flight 
If 2 or 3: 

Fidelity is equivalent to Level 2 (or 3) 

Treat same as degradation due to delays 
Disadvantages for FAA application: 

Requires a Level 1 rate response model for evaluation 
Method requires subjective pilot ratings 


MR. KEY : What will happen is if you saturate you 
will have to drive this gain down, otherwise you will be 
bumping into the stops all the time when you do 
maneuvers. 

MR. GREEN: Is that self-adaptive, though? 

MR. KEY: No, it is not. How do we set these things? 
Well, until Dick Bray retired, he did it. Now we ask him 
to do it even though he has retired. One of the motivations 
for getting these data and doing this experiment is to come 
up with a more systematic way of setting these washout 
parameters. I don’t think we have good answers yet. 

MR. HUTCHINSON: Would you like to suggest a 
time difference for the approximate cuing between the 
motion and visual? We all know that motion should pre- 
cede the visual, but do you have any specific time 
element? 

MR. KEY: You say you know the motion should pre- 
cede the visual? Well, on the VMS we could make the 
visual faster than the motion, but would have to slow the 
visual response to make the motion faster. In terms of 
pure delay, I do not think visual should be slower than any 
motion axis. Overall, it would be nice if we could get the 
phase line to lie along the aircraft model through this 


region (1 to 3 rad/sec) and increase the gain somewhat. I 
think we are trying to minimize the phase and gain distor- 
tions, that is, to minimize the gain reduction and to mini- 
mize the phase lead or lag. So whatever you can do to 
make the gain and phase of the motion and visual match 
the model is desirable. 

MR. CARDULLO: I was confused by something you 
said — that the motion was always slower than the visual, 
yet according to the numbers that you gave, in two 
degrees of freedom, the motion has actually got less delay 
than the visual. You quoted 80 msec for the visual, and in 
pitch and roll I think you quoted 70 msec for the motion. 

MR. KEY: That is true. What I thought I said was 
that we can compensate for the visual. There is a neat 
scheme for generating lead to drive the CGI. So we can 
compensate the visual down to zero. 

MR. CARDULLO: But the delay is still there; you 
just compensate the phase, essentially. You could use that 
in motion too. 

MR. KEY: No you can't. You can’t do it to the 
motion. 

DR. TISCHLER: Delay compensation will produce 
side bands at high frequency. Visual electronics is one 
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thing; in fact, in some cases even it will shudder. If we try 
to put similar lead through a motion system, I think it 
would go unstable. 

MR. CARDULLO: Is that because of the high- 
frequency anomaly that McFarland predicts? 

MR. KEY; Yes. If you take McFarland’s prediction 
and get into very high frequency inputs or turbulence, 
then things do break up. So there is a limit to the fre- 
quency range that you can use it over. And like Mark 
[Tischler] was saying, when you try to push it through a 
motion base, that frequency comes down into the usable 
range. So it can’t be done to the motion. 

MR. MITCHELL: There is lead compensation 
already on the VMS, even for those numbers. They com- 
pensated what they could to make up for delays to begin 
with. The numbers are a lot worse without lead 
compensation. 

MR. KEY : At 1 rad/sec we already have 45° of phase 
lead. 

MR. DUVAL: We experimented with a visual lead 
technique when we tied Flight Lab into the Fort Ord 
trainer last year. And what we found was that it really did 
a good job, as long as the pilot’s motion was continuous. 
But you still sense the transport delay at the onset when 
you had the first discontinuity of something abrupt. The 
lead certainly could not deal with that. Does that initial 
discontinuity affect the pilot’s perception of what’s 
going on? 


MR. KEY: Well, it sure would if it was there. But 
you were driving a different system through different sets 
of equations with the different algorithm. A lot of people 
have used this without noticing too much effect. In the 
last simulation having a high bandwidth requirement, we 
only compensated the visual down to match that motion, 
that is, 70-msec delay on the motion; we did not go all the 
way to zero. So it is much smoother. But yes, if you push 
it to far, it will get noisy. 
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17. METHODOLOGY DEVELOPMENT FOR EVALUATION OF 
SELECTIVE-FIDELITY ROTORCRAFT SIMULATION* 

MAJOR WILLIAM D. LEWIS, D. P. SCHRAGE, J. V. R. PRASAD, AND MAJOR DANIEL WOLFE 


The value of rotorcraft simulators in providing 
increased safety, reduced operating/training cost, and 
enhanced mission training has been well documented in 
the past 20 years. Because of the increased emphasis on 
rotorcraft simulation, the FAA has launched a program to 
establish certification standards for rotorcraft simulators. 
This program is aimed at updating both rotorcraft simula- 
tor standards and the methods of simulator validation 
through objective and subjective tests. No methodological 
and acceptance criteria currently exist for the performance 
and handling-qualities assessment of rotorcraft simulators. 
In order to establish certification criteria, a planned 
research effort to quantify the system capabilities of 
“selective fidelity” simulators is required. This paper 
addresses the initial step toward that goal: the establish- 
ment of a method for defining the performance and 
handling-qualities acceptance criteria for selective- 
fidelity, real-time rotorcraft simulators. Within this 
framework, the simulator is then classified based on the 
required task. The simulator is evaluated by separating the 
various subsystems (visual, motion, etc.) and applying 
corresponding fidelity constants based on the specific 
task. This method not only provides an assessment tech- 
nique, but also provides a technique for determining the 
required levels of subsystem fidelity for a specific task. 
This provides a helpful tool for use in eliminating system 
suboptimization. 

In developing a method, our task becomes twofold: 
define rotorcraft simulators in terms of fidelity and then 
apply data-collection techniques to evaluate performance 
and handling qualities. With respect to fidelity, the current 
thrust of minimizing training costs focuses attention on 
the question, What is the required level of fidelity? As a 
general rule, procurement of new simulation devices or 
the updating of existing models consisted of fulfilling a 
wish list. If a state-of-the-art system was desired, state-of- 


the-art subsystems were procured and integrated. It would 
not be inconceivable to have a high-fidelity visual and 
motion system coupled with a somewhat simplistic math- 
ematical model. After investing millions of dollars in the 
system, the pilot comments were still unfavorable, for 
example, “A very nice procedural trainer, but it just 
doesn’t fly like the aircraft.” In this case the system inte- 
grator has suboptimized the system. Unfortunately, there 
is no quantitative method for defining a required level of 
fidelity for a given simulation task. A method for assess- 
ing selective-fidelity simulators would provide the sys- 
tems integrator with acceptance criteria and would aid in 
preventing system suboptimization by defining required 
subsystem fidelity for a specific task. This paper proposes 
to approach this problem by defining a task-specific simu- 
lator classification system based on fidelity. With respect 
to applying data-collection techniques for evaluating han- 
dling qualities, ADS33, the emerging standard in heli- 
copter handling qualities, coupled with the U.S. Army 
Light Helicopter (LH) Demonstration/Validation Phase 
test results are used to define the following: 

1. Quantitative evaluation criteria. In general, data 
collection focuses on quantifiable items such as band- 
width, minimum and peak rates, and damping ratios that 
are useful in defining acceptable tolerances between 
actual flight data and simulation data. 

2. Qualitative evaluation criteria. In general, a rat- 
ing scale system for a specified set of tasks is outlined for 
pilot acceptance of the simulation. 

As depicted in figure 1 , the fidelity requirement for 
any simulation device is inherently dependent on the 
given simulation task. 

The requirements for simulators in the civil and 
military fields have expanded greatly throughout the past 
decade. Along with that growth, the variety of simulation 
tasks has also increased. Tasks can be categorized as 
follows: 


* Paper presented by Cliff McKeithan. 
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Figure 1. Fidelity dependence on type task. 


1. Non-real-time research analysis 

2. Part-task simulation 

3. Part-mission 

4. Full-mission 

5. Interactive mission scenario (networked, 
multiple nodes) 

Encompassing these tasks, simulation devices can be 
broadly categorized into three types: research, training, 
and procedural trainers. Within these broad simulation 
types, the levels of fidelity for a given type of device can 
vary greatly. For example, using cockpit crew coordina- 
tion as our simulation task, a work station can be defined 
as a relatively low-fidelity research simulator. Yet, 
another simulator of the same type, such as the Crew 
Station Research and Development Facility (CSRDF) 
located at Ames Research Center, certainly has a higher 
level of fidelity for the same task. Thus, for a specified 
task, the user must be able to determine fidelity require- 
ments. Failure to properly determine these requirements 
can result in (1) unsatisfactory results owing to a lack of 


fidelity, and (2) satisfactory results but at a premium cost 
(suboptimization). 

Consequently, it is desirable to classify a simulation 
device in terms of its fidelity. This allows a user with 
defined, task-specific fidelity requirements to select a 
simulator of appropriate fidelity and eliminate the above 
problems. The Federal Aviation Administration (FAA), 
for example, qualifies airplane training simulators in 
terms of objective fidelity. Simulator classification by 
fidelity sets a basis from which the user community can 
identify the specific simulation device that optimizes their 
needs. 

The current FAA approach to the qualification of air- 
plane simulators is embodied in FAA AC-120-40B. A 
similar approach is being planned by the FAA for qualifi- 
cation of rotorcraft simulators. The FAA approach desig- 
nates simulators in four categories, levels A through D, 
based on increasing levels of objective fidelity. Simulator 
standards, objective validation tests, and functional and 
subjective tests are then defined for each category. For 
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airplanes, the standards, validation tests, and functional 
and subjective tests have been fairly well accepted by 
industry through a series of workshops. Rotorcraft simula- 
tors do not have such well-defined standards owing to the 
unique capabilities and complexities of the air vehicle and 
existing simulation technology. Development of the 
rotorcraft criteria will require extensive research and 
development. 

Unlike the FAA approach to simulator classification, 
this method quantitatively classifies a given type of simu- 
lation device in terms of objective fidelity and a 
simulation-task-dependent weighting vector (TDWV). 
Each TDWV consists of a weighting parameter per 
fidelity characteristic, that is, 


SIMRATING task(i) = [FIDELITY CONSTANTS] 
*[TDWV] 


where 


[FIDELITY CONSTANTS] - [C Co ckpit C au dio 


^motion • ■ • ^-visual! 


[TDWV] — [Kcockpit K audio K mo ^ on . . -K visual]^" 

For example, an air-to-air combat task requires a sig- 
nificant weighting parameter for the visual characteristic, 
whereas, the instrument training task would not require as 
large a weighting parameter for the visual characteristic. 


Clearly, in general terms, the weighting vector will always 
be dependent on the simulation task to be performed. The 
fidelity of the simulation device is assessed by rating each 
component of the system. For the purposes of this 
method, a simulation device is described in terms of 
10 subsystems, with each subsystem having varying 
degrees of sophistication. 

In surveying current simulation designs and existing 
technologies, there are generally 10 subsystems which 
adequately describe a given simulation device: 


Cockpit 

Motion 

Mathematical model 
Ground handling 
System latency 


Audio 

Control system 
Environment 
Mission equipment 
Visual 


In each subsystem, it is possible to associate a level 
of fidelity with the degree of equipment/software sophis- 
tication. For example, a motion system that employs six 
degrees of freedom can be associated with high fidelity, 
whereas a fixed-base system can be associated with low 
fidelity. This association between fidelity and the subsys- 
tems defines fidelity characteristics. Subsequently, listed 
below are the fidelity characteristics (rank order; low to 
high) of the simulator subsystems that span the spectrum 
of fidelity. The fidelity characteristics are assigned 
respective values from 1 to 4. 


1 . Cockpit/crew station 

Simulated instruments 
Basic, generic-type instruments 
Partially simulated cockpit 
Full-up crew station 
3. Motion 
None 

2DOF (pitch and roll) 

3DOF (pitch, roll, and yaw) 
6DOF 

5. Mathematical model 
3 DOF 
6 DOF 

6 DOF w/simple rotor 
6 DOF w/complex rotor 
7. Ground handling 
No gear 
Rigid gear 

Simplified gear model 
Comprehensive 


2. Audio 
None 

Significant cockpit sounds 
Incidental sounds (precip., etc.) 
Realistic 

4. Control system 
No force feel 

Constant force (spring/damper) 
Partial duplication of actual force 
Complete duplication 
6. Environmental 
Clean air 
Discrete gusts 

First-order filtered turbulence 
Rotationally sampled turbulence 
8. Mission equipment 
None 

Communication only 
Communication/navigation only 
Complete 
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System latency 

10. Visual 



Non-real-time (off line) 

None 



Significant delay 

Field of view 

Dvnamic ranae 

Detail 

Minimal delays 

Workstation 

Day 

Low 

Real time 

75° horiz./30° vert. 

Dusk 

Medium 


90° horiz./40° vert. 

Haze/fog 

High 


Wider 

Night 

Very high 


Assigning a value to each fidelity characteristic of the 
simulation device allows us to quantify fidelity by form- 
ing the fidelity constants matrix. For example, the U.S. 
Army 2B38 UH-60 simulator has the following 
characteristics: 

1. Cockpit: full up crew station 

2. Audio: incidental sounds 

3. Motion: 6DOF 

4. Control system: complete duplication 

5. Math model: 6DOF w/simple rotor 

6. Environment: discrete gusts 

7. Ground handling: simple gear model 

8. Mission equipment: complete 

9. System latency: real time 

10. Visual: 90° horiz./40° vert, full dynamic range 
medium detail 

With the above characteristics, the UH60 training 
simulator’s fidelity constants matrix is 

[FIDCONST] = [C coc kpjt, C auc j, C mo t, Cf ee ], C ma th. 

C e nv> Cgmd> C m ep> Q a L Cvis] 

= [4 3 44 3 2 344 3.25] 

For a given simulation task, minimum acceptable 
fidelity characteristics must be established in order to 
constrain the number of simulation devices eligible to per- 
form the task. For example, to conduct aircrew contact 
training, some form of visual system is a minimum 
requirement for the visual fidelity characteristic. Without 
a visual system, the device would be unable to adequately 
provide task training. Consequently, a FIDCONST m i n 
matrix: 

| FIDCONSTmin] = {min[C toc kpjt C au( jio 
Cmotion • ■ • Cvisuall) 

is utilized to establish the minimum acceptable fidelity 
characteristics for a given task. Exemplifying this concept, 
the U.S. Army 2B24 instrument training simulator, 
although it has many high-fidelity characteristics, such as 
a 6DOF motion system, full-up cockpit, and a complete 


mission package, is not eligible for consideration as a 
simulator for contact training because it lacks a visual 
system 

The function 

SIMRATING,ask(i) = [FIDCONST] * [TDWV task(i) ] 
constrained by 

[FIDCONST m j n ] = {min[C cock pjt C aU( jj 0 
Cmotion • • • C visual]} 

permits classification of a type-simulation device with 
respect to fidelity. Given a simulation task, a 
FIDCONST m j n matrix and a TDWV are determined, 
either subjectively or through extensive research. Once 
the weighting vector is known, a minimum and maximum 
SIMRATINGtask(i) is calculated. Given this range of val- 
ues, the simulation devices can be classified in terms of 
fidelity for a specified task. The range of values is parti- 
tioned into five subranges, the lowest corresponding to 
poor fidelity and the highest corresponding to high 
fidelity. 

As an example, suppose the given task is instrument 
training and the hypothetical FIDCONST m j n and TDWV 
have been determined to be 

[FIDCONST min ] =[4 244321341] 

[TDWV] = [ 1 0.5 1 1 0.75 0.5 0.25 0.75 1 0.25] 

Multiplying [FIDCONST min ]*[TDWV] we find the min- 
imum SIMRATING tas k(i) to be 23. For the maximum 
SIMRATINGtask(i)- we must multiply 

[FIDCONST ma xl*[TDWV] 

where the maximum fidelity constant matrix 
[FIDCONST max ] is defined as 

[FIDCONST max ] = [4 4 ... 4] 
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Thus, the maximum SIMRATING tas k(i) is calculated to 
be 112. Partitioning this range of values, we can now form 
a task specific (instrument training) classification for sim- 
ulation devices based on Fidelity. For this example: 


Fidelity 

Classification 

SIMRATING 

Excellent 

A 

94-112 

High 

B 

76-93 

Medium 

C 

58-75 

Low 

D 

41-57 

Lowest 

E 

23-40 


Within the scope of this method, the fidelity charac- 
teristics were limited to a range of 1 to 4 in order to pro- 
vide an equivalent weighting between characteristics. This 
general approach obviously cannot handle specifics of any 
single characteristic. An alternative approach to provide 
equivalent weighting between characteristics is to employ 
normalized matrices for each characteristic. This approach 
would allow a greater degree of flexibility in assessing 
each characteristic. For example, while assessing the 
visual system the user could include the use of texture, 
infinity collimation, display types, etc. over a wider range 
of values. This enables the visual system characteristic to 
be well defined in terms of its specific attributes. 

The approach has assumed fidelity constants. This 
implies that no coupling exists between the various 
fidelity characteristics. Anyone who has flown in a simu- 
lator with a high-fidelity visual system employing infinity 
collimation knows this to be untrue. With a fixed-base 
motion system, the aforementioned visual system will 
cause a perceived motion. The strength of the perceived 
motion will vary, depending on the fidelity of the visual 
system. This example would indicate some degree of cou- 
pling between the visual and motion characteristics. This 
interdependance may be better represented by use of a 
matrix. The terms of the matrix could be constants or 
variables. The exact form of the coupling would need to 
be determined through research. 

The method at this point allows categorization of 
rotorcraft simulators in terms of fidelity for a specific 
task, but leaves unanswered the means of evaluating the 
performance and handling qualities of the rotorcraft 
simulator. 

A simulator must be assessed in the areas critical to 
the accomplishment of the assigned mission task. These 
areas typically include longitudinal- and lateral- 
directional responses, performance in takeoff, climb, 
cruise, descent, etc. Objective tests are used to quantita- 


tively compare simulator and aircraft data to ensure that 
they agree within some specified tolerance. ADS33 speci- 
fies an absolute standard for actual rotorcraft stability 
behavior. Requirements for handling-qualities standards 
are quantitatively specified, often in terms of frequency 
responses. Subsequently, characteristics of frequency 
response, such as bandwidth, damping ratios, overshoot, 
and time-to-peak become the tools of quantitative evalua- 
tion criteria. The method of ADS33 is applicable to simu- 
lation as well, except now these quantitative tools define 
tolerances between flight-test data and simulation data. 

Historically, simulator performance has been evalu- 
ated in terms of the simulator’s original design specifica- 
tion. This specification normally requires the simulator 
designer to meet the aircraft’s flight-test data within 
specified tolerances. Paralleling the FAA’s approach, per- 
formance testing will include the following flight regimes: 
hover, vertical and forward flight climb, level flight, and 
autorotational descent. The method of performance testing 
will consist of classic test techniques as outlined in 
USNTPS-FTM-106, Rotary Wing Performance, refer- 
ence x. Tolerances between actual and simulated flight 
data are then established for each phase of flight based on 
simulator category. The tolerance for a category A simu- 
lator is thus the most restrictive and the tolerance for a 
category E simulator is the most relaxed. Figure 2 illus- 
trates the relation between the level of tolerance and the 
simulator category. The level of tolerance, represented by 
the expanding circles, reflects an increasing ± tolerance 
range with decreasing simulator fidelity classification. 
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Although classic performance testing techniques were 
adequate for evaluation purposes, classic handling-quali- 
ties testing techniques do not provide adequate informa- 
tion for assessing comparative simulator response. For the 
past 8 years, the U.S. Army, with participation from the 
other military services, the FAA, and industry, has been 
developing a new approach to specifying flight-handling 
qualities for rotorcraft. The existing military specification, 
MIL-H-8501 A, was first published in the early 1950s and 
was revised once in the early 1960s. The new specifica- 
tion will eventually be designated MIL-H-8501 B; how- 
ever, for application to the U.S. Army LH procurement, 
the designation ADS33 has been issued. The approach in 
this new specification is based on defining mission task 
elements (MTE’s) and relating the visual cue environment 
(VCE) experienced in the aircraft to the level of stabiliza- 
tion required. Although the approach is currently being 
applied to qualifying rotorcraft, it will have substantial 
applicability to rotorcraft simulators. ADS33 provides 
clear quantitative requirements for classifying rotorcraft in 
terms of their handling qualities. A designation of 
levels (I, II, III) is utilized. These requirements are 
divided into three main categories; control-system charac- 
teristics, hover and low speed, and forward flight. Apply- 
ing this same standard to simulation, these categories now 
define evaluation criteria for simulation devices. Subse- 
quently, a set of tolerance levels between flight and simu- 
lation data must be established for each simulator cate- 
gory as described in figure 2. A set of flight-test maneu- 
vers based on mission-task elements is simulated to obtain 
quantitative and qualitative data. These quantitative data 
are then analyzed, and a comparison with actual flight-test 
data is conducted. The deviation between actual and simu- 
lated flight data then becomes the measure of acceptabil 


ity. The proximity to the specified tolerance then validates 
the simulation device classification. 

Pilot acceptance is a subjective evaluation. Subjective 
tests are designed to provide a basis for evaluating simu- 
lator capability to perform over a typical training period 
and to verify correct operation of the simulator instru- 
ments and systems. With respect to ADS33, the flight 
maneuvers outlined in the previous paragraph serve as the 
vehicle for a subjective, qualitative evaluation. Based on 
mission-task elements and the visual-cue environment, 
this set of flight maneuvers allows the pilot to assess the 
perceived performance and handling-quality characteris- 
tics of the simulator. These are then compared with the 
pilot’s assessment of identical maneuvers in the aircraft. 
This set of flight maneuvers allows the pilot to explore the 
perceptual fidelity of the system so that a fair assessment 
can be made. A Cooper-Harper rating scale system is used 
for the evaluation. 

Conclusion 

The method discussed here offers the rotorcraft simu- 
lation community a unique tool for analyzing and tailoring 
simulation devices for specific requirements. By tying 
fidelity directly to the simulation task, linkage is achieved 
through the simulator classification model. Concurrently, 
methods for evaluating quantitatively and qualitatively the 
performance and handling qualities of a rotorcraft simula- 
tion device are presented. These methods are consistent 
with current evaluation criteria. Additionally, this 
approach permits melding of the FAA certification 
method with the emerging rotorcraft handling-qualities 
specification, ADS-33. 
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APPENDIX A 


PANEL DISCUSSION SUMMARIES 
NASA/FAA HELICOPTER SIMULATOR WORKSHOP 




FOREWORD 


These summaries were developed from transcriptions of stenographic recordings of the session presentations and 
discussions. Although care has been taken to identify and eliminate the errors of interpretation to which this technique is 
prone, undoubtedly others remain. They are the responsibility of the editor; the discussion moderators and panelists are 
blameless, at least in this regard. 


1. Session A: Training: Limits, Allowances, and Future 

Ronald J. Adams 

2. Session B: Scene Content and Simulator Training Effectiveness 

Walter W. Johnson 

3. Session C: Low-Cost Training Alternatives: Part- and Full-Task Trainers 

David A. Lombardo 

4. Session D: Dynamic Response and Engineering Fidelity in Simulation 

Edward D. Cook 

5. Session E: Current Training: Where Are We? 

Greg J. McGowan 

6. Session F: Aero Modeling 

Ronald W. Du Val 
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1. SESSION A 


TRAINING: LIMITS, ALLOWANCES, AND THE FUTURE 

RICHARD J. ADAMS,* MODERATOR 


Panelists. Edward Boothe, FAA National Simulation Program Office; Martin Flax, Northrop Corporation; Edward 
Stark, Research Consultant; Curt Treichel, United Technologies, Inc. 


Principal Topics. Improved training and safety using simulators; regulatory limitations on testing helicopter emer- 
gencies; reduced training and cost; recommended aeronautical experience flight proficiency regulatory changes; certifi- 
cation credit for improved simulator training; and working-group proposed revisions to airman certification regulations. 


Historically, the qualification, approval, and use of 
helicopter simulators have been constrained by the state of 
the art of visual-system fidelity and phase lag or motion- 
system performance. The effects of these technological 
limitations on the low-speed performance and hover char- 
acteristics of simulators have been to curtail the use of 
simulators for airman certification purposes. The intent of 
this session was to develop a statement of user needs for 
simulators, to analyze the skills pilots need to do their 
jobs, and to examine the suitability of presently available 
simulators and motion and visual systems. 

Rather than dealing specifically with the principal 
topics originally suggested for Session A, the panelists 
encouraged a wide-ranging, open discussion as a means of 
getting ideas presented and discussed and eliciting com- 
ments and criticisms. 

Warren Robbins (FAA Flight Standards) reviewed 
the proposed Title 14, Code of Federal Regulations, 

Part 142. The proposed Part 142 provides for a much 
more extensive use of helicopter simulators and of other, 
various-level training devices. Given this new regulatory 
sanction, it will remain for each affected school to 
develop an appropriate training program and associated 
syllabi. Once the programs and syllabi are approved, the 
schools will be free to market their products. Ed Boothe 
also pointed out that the new Part 142 certified training 
school would afford much greater flexibility in the ways 
in which helicopter simulators could be used. 
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The Session A group agreed upon four recommenda- 
tions. First, it agreed to support Ed Boothe in his efforts 
to bring out the advisory circular. This support would 
extend beyond the workshop to provide support for the 
philosophy underlying the circular. 

Second, it was agreed to support changes in the pro- 
posed MPRM for Part 142 when that document is made 
available for public comment. Warren Robbins is making 
every effort to produce a good and useful document, and 
the timely submission of panel comments will facilitate 
his work. 

The third recommendation had to do with exemp- 
tions, in particular with supporting and encouraging Greg 
McGowan in his pursuit of additional exemptions that 
would enable the further utilization of the FlightSafety 
simulators. Special emphasis was placed on his attempts 
to gain approval of the simulator for use in granting add- 
on ratings. Some of those present agreed to work with 
Greg in producing another letter requesting this latter 
exemption. Greg said that FlightSafety is presently seek- 
ing an addendum or change to 4609 that would allow 
them to do some of the things discussed at the workshop. 
Other efforts along this line are more or less on hold, 
pending the outcome of this request. 

The near-term plan in this regard is to request as 
many exemptions as practical while supporting longer- 
term objectives. The issue of treating the simulator as a 
training tool, just as the aircraft is, generated extensive 
discussion. It is recognized that both these training 
devices have limitations, but it is important to recognize 
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that both have real and useful capabilities that should be 
appropriately exploited. 

It would be desirable if the regulations would permit 
rating approval, testing, and licensing approval for simula- 
tors, if that is what the student has access to, or for the 
aircraft, if that is available to him. If he fails in the simula- 
tor, however, he fails just as certainly and to the same 
extent as if he had failed in the aircraft. 

Fourth, it was recommended that the simulator be 
used as a crew training and evaluation tool. Because 


industry is moving more and more toward the use of 
simulators and because interpersonal skills and resource 
management are key safety factors, it was agreed that 
these skills could best be evaluated in a simulator. 

Although Session A discussions did touch on the 
issue of levels of sophistication that simulators would 
have to possess before flight-hour credits could be given, 
this matter was not considered in detail and was left for 
future meetings. 



2. SESSION B 


SCENE CONTENT AND SIMULATOR TRAINING EFFECTIVENESS 


WALTER W. JOHNSON,* MODERATOR 


Panelists. Robert Hennessy, Monterey Technology, Inc.; Robert J. Randle, NASA Ames Research Center. 


Principal Topics. Scene display technology; scene image content; simulator utilization; and compliance evaluation. 


The single most important feature of modern flight- 
training simulators is their visual systems, but relatively 
little work has been done to determine precisely what 
scene content best supports the training functions they 
perform. 

The emphasis has been on how well scenes are drawn 
(resolution) and on how fast they are drawn (update rate). 
The principal purpose of Session B was to promote a dis- 
cussion of how the physical scene presented by the 
simulator — for example, terrain, clouds, and objects — can 
influence the effectiveness of simulator training. 

The panel addressed two main issues related to scene 
display technology in helicopter simulators: the impor- 
tant ways in which this technology can affect depth per- 
ception, and minimal field-of-view (FOV) requirements in 
simulators. 

For performing low-level, close-in missions, as heli- 
copters are often required to do, appropriate depth cues 
are viewed as being of major importance. In this regard, 
both collimated displays and the absence of binocular dis- 
plays were discussed. Collimated optics, which cause all 
displayed objects to appear at a great distance, generate a 
compelling feeling of depth in the displayed scene. How- 
ever, this is optically correct only for simulating objects 
that are far way from the observer, and thereby conflicts 
with scene content information — for example, perspective 
and absolute size — in which the objects are shown at 
shorter ranges. Consequently, there were recommenda- 
tions by panel members that ( 1 ) a thorough analysis of 
image collimation be conducted to determine how it 
affects or distorts the appropriate optics for near-objects, 
with particular attention to different eye positions 
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(pilot/co-pilot) or observer head movements; and 
(2) human performance studies be undertaken to evaluate 
the importance of these depth cues and their accompany- 
ing distortions. 

The need for good binocular cues was considered, 
with many panel members saying these cues are essential 
in low-level, close-in tasks. It was pointed out that we are 
capable of testing the importance of binocular cues in 
many head-slaved systems, but have not yet done so. 
Because providing binocular cues will, of necessity, 
require head-mounted displays, a significant cost will be 
incurred. Nonetheless, some of the researchers involved in 
this work consider the provision of binocular cues a 
potentially critical factor in close-in work capability for 
helicopters. 

Field-of-view (FOV) requirements are considered to 
be an essential issue by industry and research workers. 
Because displays are the major cost items in simulation 
systems, industry needs to know what the requirements 
are. The panel did not find a consensus on this matter, but 
several related points were brought up during the 
discussions. 

It was agreed, for example, that FOV requirements 
are largely maneuver-dependent, and that the horizontal 
FOV must exceed 140°, although the need for FOVs 
greater than 180° was questioned. It was also noted that 
although many pilots want a vertical FOV, such a capa- 
bility is often unavailable in the actual aircraft. Moreover, 
vertical field of view is most often a function of cockpit 
design, but is also dependent on the rolling and pitching 
that are often encountered during maneuvers. For exam- 
ple, during decelerating landings, the helicopter often 
pitches up, thus eliminating any forward views looking 
downward, even though the cockpit design allows such a 
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view in other orientations. Consequently, the important 
thing is to make the FOV in the helicopter simulator 
appropriate to the helicopter and to the maneuvers being 
simulated. 

The panel discussion of scene image content was 
concerned principally with the issues of realism and gen- 
eralization. The FAA representative wants the scene to be 
as realistic as possible, the reason being to make the simu- 
lator capable of doing final check rides, especially for 
approaches into urban vertiports, and of allowing the pilot 
to fly as well in the simulator as he would in the heli- 
copter. In this regard, there were discussions about scenes 
depicting specific areas (e.g„ a vertiport in a given city) or 
if the effort should be, instead, to present general charac- 
teristics from a range of possibilities. 

Unfortunately, the desire for maximum realism is in 
conflict with training uses of the simulator, in which the 
ability to use both generic visual cues (horizon ratios, tex- 
ture density, known size scaling) and special cue training 
paradigms was considered of utmost importance. Again, 
the panel concluded that the lack of essential research into 
some of these topics made it difficult to establish a visual 
data base that would at once be optimum for training and 
for efficient pilot certification testing. 

In the panel’s discussion of the utilization of simula- 
tors it was noted that training (initial, transitional, and 
recurrent) and certification applications of helicopter 
simulators may well require significant differences in the 
visual scenes used. For example, training effectiveness is 


often improved by selective manipulations of the visual 
scene, whereas certification testing requires highly stan- 
dardized formats. 

The panel members acknowledged that certification 
of compliance is a difficult issue. Some thought that the 
introduction of compliance requirements for the visual 
scene data-base design would result in prohibitive cost 
increases. As a result, it was proposed that consulting 
groups of experts should be the recommended approach. 

It was also proposed that methods should be devel- 
oped for evaluating compliance; expert opinion is an 
example of this approach, but performance-based criteria 
were also suggested. The point was made that it is diffi- 
cult to have principled compliance criteria without mea- 
surements of in-flight pilot performance as a basic 
reference. 

FAA representatives involved in TERPS develop- 
ment want simulators to permit performance as good as 
that that can be achieved in flight, thereby ensuring that 
TERPS criteria can be met during certification flights. 
However, others cautioned that this performance must not 
be achieved by making the simulation unrealistic. 

In general, the panel members agreed that the tech- 
nology exists to provide the visual scene content that is 
required in simulations, but that we do not yet know what 
we should put in the visual data bases. Similarly, the dis- 
play technology required for close-in helicopter missions 
has not been explored. Both of these deficiencies must be 
addressed in a more direct manner. 
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3. SESSION C 


LOW-COST TRAINING ALTERNATIVES; PART- AND FULL-TASK 

TRAINERS 


DAVID A. LOMBARDO,* MODERATOR 


Panelists. Graham Beasley, Silicon Graphics; Jack Dohme, U.S. Army Research Institute; Steve Hampton, Embry- 
Riddle Aeronautical University; Alfred Lee, Integrated Systems Engineering. 


Principal Topics. Personal computers and training software; computer-based flight-training devices; building the 
modular simulator; and designing for training device effectiveness. 


The purpose of Session C was to review the back- 
ground of and current research efforts in the general area 
of low-cost, computer-based simulation alternatives, as 
well as to provide recommendations for directing future, 
related work. Low-cost simulation alternatives are defined 
to include computer-based flight simulation, and both 
generic and type-specific non-motion flight-training 
devices. These alternatives have been used by many 
fixed-wing operators with great success, but have been 
otherwise ignored because they do not meet FAA guide- 
lines for flight-training devices and simulators. Nonthe- 
less, technological advances in the microprocessor indus- 
try ensure that the training capabilities of these and similar 
devices will be moved forward in directions and ways that 
are as of now unimaginable. 

Session C panel members from Silicon Graphics, the 
U.S. Army Research Institute at Fort Rucker, the Embry- 
Riddle Aeronautical University, Integrated Systems Engi- 
neering, and Bowling Green University presented a series 
of wide-ranging papers. There were discussions, among 
others, of what constitutes a “low-cost” alternative, and 
what can be expected of them in terms of capability. 

Research concluded earlier this year — in which 
computer-based training and computer-based flight simu- 
lation and their applications to teaching instrument proce- 
dures and, primarily, navigation procedures (e.g., VOR, 
ADF) — was reviewed. Overviews of low-cost training 
devices and a summary of a project in which computer- 


based training was specifically applied to attitude-instru- 
ment flying were presented. In the latter, emphasis was on 
ab initio students and the extent to which they could be 
effectively trained in attitude instrument flying through 
use of a computer-based training program. 

Regarding personal computers and training soft- 
ware, there was a consensus that they are worthy of addi- 
tional support. Panel members viewed them as a develop- 
ing technology, a way of the future. Virtual reality was 
discussed, and what is viewed as its major implications 
for simulation was summarized. 

The panel’s discussion of computer-based flight- 
training devices was never developed owing to time con- 
straints. In discussions of the modular simulator, there 
was general agreement about a generic type of data base 
and that a reasonable amount of vertical information 
would have to be presented for effective helicopter train- 
ing. The discussion here dealt principally with the data 
base itself, with only limited consideration given to such 
items as the number of channels required and the ones 
used the most. 

The fourth topic for the panel’s consideration was 
designing for training device effectiveness. In brief, the 
discussion of this topic reduced itself to a question: Are 
the data available that would permit a reliable prediction 
of the training effectiveness of a given simulator without 
the need of evaluating the simulator? The panel concluded 
that the answer is no — the data do not exist. 
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4. SESSION D 


DYNAMIC RESPONSE AND ENGINEERING FIDELITY IN SIMULATION 


EDWARD D. COOK,* MODERATOR 


Panelists. Richard Bray, NASA Ames Research Center; Roger Hoh, HOH Aeronautics, Inc.; A1 Sodergren, FAA. 


Principal Topics. Maximum tolerable transport delay; handling-qualities parameters; other matching parameters; 
motion parameters; and visual system parameters. 


Simulator responses to control inputs must duplicate, 
within specified tolerances, the responses that the same 
inputs would effect in the actual aircraft. It is these 
tolerances that constitute the main subject matter in the 
development of simulator standards. The responses that 
are usually measured are of three general kinds: (1) air- 
craft responses of the kind frequently used to measure 
handling qualities; (2) limiting transport delays; and 
(3) correct motion responses. These responses have long 
been used to ensure adequate simulator fidelity so that 
pilot skills learned in the simulator transfer to aircraft. The 
application of these methods to helicopters re-opens the 
issues discussed in Session D. 

Regarding transport delay, the panelists first dealt 
with the question of what constitutes an acceptable delay. 
The consensus seems to be for a transport delay of about 
100 msec. There are commonly used methods for reduc- 
ing the effects of transport delay by adding lead to the 
system. Which brings up the question of how to check the 
efficiency of such methods. Should it be done in the fre- 
quency domain? For example, should it be done using a 
sine wave or with a step input? The problem is that delay 
can be compensated for with a lead circuit in the fre- 
quency domain only so long as there is a fairly smooth 
and continuous input. 

Given a sudden step, however, there will still be a 
temporary delay in the transport delay. One panel member 
recommended that the phase delay parameters used in 
handling-qualities analyses are potential metrics for 
determining whether the simulator properly represents its 
stability characteristics to the pilot as a whole. And this 
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leads to the question of defining appropriate handling- 
qualities parameters and selecting the correct ones to use. 

The panel agreed that the frequency-response data of 
the total end-to-end system are probably as good a crite- 
rion as any for determining whether the system is working 
together as a unit and whether it represents itself to the 
pilot as the real system it simulates. 

Do the Cooper-Harper ratings serve as a good basis 
for comparisons? In terms of validating a simulation, is 
the practice of having a pilot rate both the simulation and 
the flight vehicle a reasonable one? The panel decided 
that the answer is probably no. Comparisons of that kind 
are seen as being too time-consuming and too costly. 

Concerning which motion parameters should be 
specified, the panelists agreed that the bandwidth of the 
motion hardware, which is a limiting factor, has to be 
increased. If it is not possible to do a good, all-around job, 
the yaw axis, vibration, and on-ground contact were 
selected as being the most important motion cues and the 
ones that should be emphasized. As for latency in the 
visual system, it can, to a great extent, be corrected by 
prediction techniques. The sudden step will still cause 
delay, but when the motion is continuous, the delay can be 
led and tracked. 

It was agreed that the actual latency that must be cat- 
egorized should be task-dependent and driven by the stip- 
ulated level of certification. That is, there should be no 
one generalized number. And a final comment — there 
were suggestions that a 30° field of view downward 
through the chin window should be provided. 
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The panel discussion also touched on the problem of talk about providing subtle augmentation in the simulation 
the relative difficulty of flight in the simulator and in the as a means of making the workloads in the simulator and 
aircraft. For example, hovering is probably more difficult aircraft more equivalent, 
in the simulator than in the aircraft, and there was some 
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5. SESSION E 


CURRENT TRAINING: WHERE ARE WE? 


GREG J. MCGOWAN,* MODERATOR 


Panelists. Kenneth Cross, Anacapa Sciences; Gerald Golden, Petroleum Helicopters, Inc.; Douglas Schwartz, 
FlightSafety International. 


Principal Topics. Areas of simulation improvement; simulation utilization; and economics and accessibility. 


The use of commercial helicopter simulators for 
training and checking is controlled by FAA regulations 
(FARs). In some cases, exemptions to the FARs are 
granted for the use of an approved simulator; in others, 
the FARs themselves permit use of the simulator for spec- 
ified training and checking procedures. However, many 
procedures — especially emergency procedures — that are 
routinely practiced in simulators, are not required by the 
FARs. As a result, the simulator’s capacity for training 
that goes beyond the scope of the FARs is being 
underutilized. 

The Session E panel convened to discuss, in general, 
the three principal topics mentioned above — areas of 
simulation improvement, simulation utilization, and the 
economics and accessibility of helicopter simulators. In 
the event, however, the discussion centered on the third of 
those topics, the economics and availability issue, modi- 
fied, however, to couple the “helicopter simulator” term 
with “training devices.” 

Most of this panel’s discussion pertained to training 
devices and to what can be done, especially for devices 
that rank in capability (and thus in complexity and cost) 
below the approved simulator level, to enhance their 
availability and to make them more economical to use and 
maintain. 

The panel agreed that the benefits of helicopter simu- 
lation training can be made economically available to a 
larger segment of the helicopter-user community only 
through use of training devices that offer a range of train- 
ing capabilities. Two prerequisites to ensure that such an 
expansion in training-device availability occurs were 
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identified: (1) definition of the training objectives and 
tasks that these devices would or should address, and 
(2) the development of an advisory circular, or of an 
appendix to an existing advisory circular, that would set 
forth the criteria with which the various levels of training 
devices would have to comply. 

A third step, an outgrowth of item (2) above, identi- 
fies the need to establish the training and checking 
requirements that will be allowed for each level of train- 
ing device. That is, a determination must be made about 
how these devices will be certified and about the training 
uses for which they will be approved. 

These are essential considerations for the training- 
device manufacturers. If these devices are going to be 
made available to the operators, the manufacturers must 
have assurance that the devices they propose to produce 
are going to meet preestablished criteria of acceptability. 
If such criteria are not set forth, the risk of manufacturing 
the devices is too great to be entertained. 

In the panel’s discussion of the above, another ques- 
tion surfaced: Would the envisioned proliferation of 
lower-level training devices act to stifle the development 
of the more elaborate and technologically superior 
simulators — the Level C and D simulators? The panel’s 
answer was no — the widespread use of limited-capability 
but effective training devices will not cut into the market 
for the highly capable machines. On the contrary, the 
panel finds it likely that the lower-level training devices, 
by introducing operators to the possibilities of simulation 
training in general, will act as a market stimulant for the 
more advanced (and expensive) simulators. As more and 
more operators use the lower-level devices, their interest 
in the higher-level ones will be heightened and they will. 
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perhaps, come to constitute a new market segment for the 
full-fledged simulator. 

Although not an agenda item for this session, the 
panel discussed the issue of transfer of training. As a 
result, it recommended a thoroughgoing review of all 
studies pertaining to the transfer of training from simula- 
tor (or training device) to the actual flight vehicle. That is, 
does a skill mastered on the training device transfer posi- 
tively and directly to operation of the aircraft? All sources 
of such information should be exploited — government, 
military, commercial, domestic or foreign. 

If the review discloses that the information on trans- 
fer of training is inadequate for purposes of making reli- 
able conclusions, the panel recommended that an appro- 
priately designed study, one of adequate scope to ensure 
comprehensive data production, be conducted. It is a 


given that transfer-of-training studies are difficult and 
expensive. Consequently, it is the panel’s suggestion that 
such an effort be undertaken with the full cooperation of 
the government, industry, operators, and users. 

In conjunction with any transfer-of-training study, 
there is a need for a well-defined helicopter job-task 
analysis. A previously conducted job-task analysis identi- 
fied 56 jobs that are now being done with helicopters. The 
panel’s recommendation is that these helicopter jobs be 
analyzed and broken down into their component tasks. 
Then, given the results of a comprehensive transfer-of- 
training study, the most effective training devices or 
simulators can be matched with the training needs at hand 
to produce the most effective and economical training. 
Task-designated priorities would ensure that tasks having 
the most direct bearing on safety would be addressed first. 
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6. SESSION F 


AERO MODELING 


RONALD DU VAL,* MODERATOR 


Panelists. Frank Cardulio, State University of New York; R. Thomas Galloway, Naval Systems Training Center; 
Robert Toller, Quintron; Gary Hill, NASA Ames Research Center, 


Principal Topics. Physically based simulation models; validation of physically based models; and achieving a 
higher level of physical modeling simulation. 


Trainer manufacturers typically relay heavily on 
empirical models as a means of reducing computation 
time and maximizing tunability. Unfortunately, these 
models may provide poor fidelity away from the test 
points, and this is particularly true of rotorcraft simulators, 
in which empiricism may mask additional degrees of 
freedom as well as severe nonlinearities. This panel’s 
purpose was to consider the need for an increased level of 
physically based modeling in rotorcraft simulators. 

The panel’s discussion centered on the trade-offs 
between physical and functional modeling for training 
simulations. It became clear during the discussion that 
terms had to be better defined, and from that evolved a 
better understanding of what is meant by an acceptable 
form of functional simulation: one traceable to first prin- 
ciples through a physical simulation. 

For example, there are instances in which a rotor-map 
model may be an appropriate simulation model. And as 
long as the rotor-map model is traceable to a blade- 
element model from which it was derived, the 
functionality can be traced. That is to say, if at first a 
physical model of the system is created, and if the 
necessary approximations and reductions are made to 
bring it down to an appropriate level for the task to be 
undertaken, it should then be possible to track it back to 
the higher-level engineering model; in that way, control 
can be maintained over the procedures used to provide the 
modeling. 

The other level of traceability is through experimental 
data; for example, modeling an airfoil in terms of lift and 
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drag data that are traceable to a wind-tunnel test. The 
point is that the level of functionality or analytical model- 
ing present at any point in the simulation has to be depen- 
dent on the tasks performed on the simulator and on the 
level of certification that the simulation is intended to 
support. But for comparative purposes, it should be pos- 
sible to trace any functional representation to a higher 
physical level so that the assumptions involved and the 
conditions under which the functionality is valid can be 
known. 

The trade-offs concerning rotor-map and blade- 
element models were considered in this session, and it 
was concluded that the magnitude of the computational 
task associated with the blade-element model is no longer 
a significant limitation in its application to training 
simulation. Although in the past the computation costs of 
the blade-element model were prohibitive, fast parallel- 
processing computers are available and are up to the 
computational tasks involved. As a result, decisions 
concerning the use of one or the other of these models 
should no longer be based on computation costs. 

Choosing between rotor-map and blade-element 
models means considering model tunability. The rotor- 
map model is easier to tune as a means of complying with 
acceptance criteria, but whether that is desirable or not has 
not been resolved. If model performance is force-fitted to 
comply with acceptance criteria, it is no longer a physi- 
cally based model, and its validity between test points is 
unknown. Tuning the blade-element model, on the other 
hand, requires validation from physically meaningful 
parameters or from model structure changes, which is a 
much more costly process. 
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Interactional aerodynamic and inflow models were 
viewed as comprising an important problem area, but one 
that is commonly neglected in training simulations. The 
empirical models that are used to cover these problems 
are often inadequate. It was noted that there is a require- 
ment for shipboard landing simulations that can properly 
account for the aerodynamic interactions of the rotor wake 
during approaches to rolling ship decks and for the inter- 
ference of the ship superstructure with the aircraft. Even 
engineering-level simulations lack adequate modeling to > 
properly assess these issues. The solutions to some of 
these problems await technological developments. 

Other issues involve solution and integration tech- 
niques. Although not usually set out in the acceptance test 
criteria, these factors nonetheless significantly affect 
simulator performance. The questions here are whether 
degrees of freedom are to be solved simultaneously or 
sequentially, how large an integration step size to use, and 
what kind of integration algorithms to use. The alterna- 
tives are many, perhaps to such an extent that they con- 
tribute to the problem — there are so many different 
approaches that can be pursued. 

The panel’s discussions emphasized the advantage of 
subsystem-by-subsystem validation over complete end-to- 
end validation of the entire system. With the former, what 
is required and how it is required can be stated more 
specifically. Instead of looking at the aircraft response to 
stick movement, for example, one looks at the way the 
rotor responds. Isolating the various components of the 
simulation model and validating them individually, 
improves the flexibility with which simulation models can 
be interchanged in future machines. As a result, one 
would not have to start anew with each simulation. More- 
over, there would be greater confidence that the model 
was correct off test points. 


The way models are validated also affects the physi- 
cal model. A simulation may begin with a lot of physical 
content but then have a whole structure of tuning coeffi- 
cients superimposed on it when it comes to meeting the 
acceptance test criteria. At present, procurement spec- 
ifications do not prohibit the manufacturer from using this 
means of passing the acceptance test. So perhaps consid- 
eration should be given to specifying which parameters 
can be tuned, thus making certain that it is done in a phys- 
ically meaningful manner. 

How can contractors be required to use a higher level 
of engineering analysis and fidelity in their training mod- 
els? One way would be to specify that each subsystem be 
validated separately and to specify the acceptance tests in 
terms of frequency-domain criteria. For example, specify- 
ing the frequency response of the rotor with respect to 
motions of the hub would mandate a blade-element 
model; accurate frequency-response data for the rotor 
could not be achieved with a rotor-map model. 

A final and valid question that came out in the 
panel’s discussion: Why create fine physically based 
models when the control system completely overwhelms 
the physical aspects of the system? It is true that the pilot 
cannot appreciate what is going on because of the heavy 
suppression of the control system. This leads to another 
question: Should a simulation be validated only for the 
nominal flight condition — control system on in the middle 
of the envelope with mild maneuvering — or should 
training systems be validated to properly model extreme 
conditions? If the latter is the goal, that is, if control- 
system failures, edge-of-the-envelope maneuvering, and 
other aggressive maneuvers are to be modeled, the 
mathematical basis for the simulation has to be far more 
sophisticated. 
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